{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolutional Neural Network\n",
    "\n",
    "In this second exercise-notebook we will play with Convolutional Neural Network (CNN). \n",
    "\n",
    "As you should have seen, a CNN is a feed-forward neural network tipically composed of Convolutional, MaxPooling and Dense layers. \n",
    "\n",
    "If the task implemented by the CNN is a classification task, the last Dense layer should use the **Softmax** activation, and the loss should be the **categorical crossentropy**.\n",
    "\n",
    "Reference: [https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py]()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training the network\n",
    "\n",
    "We will train our network on the **CIFAR10** [dataset](https://www.cs.toronto.edu/~kriz/cifar.html), which contains `50,000` 32x32 color training images, labeled over 10 categories, and 10,000 test images. \n",
    "\n",
    "As this dataset is also included in Keras datasets, we just ask the `keras.datasets` module for the dataset.\n",
    "\n",
    "Training and test images are normalized to lie in the $\\left[0,1\\right]$ interval."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import cifar10\n",
    "from keras.utils import np_utils\n",
    "\n",
    "nb_classes = 10\n",
    "\n",
    "(X_train, y_train), (X_test, y_test) = cifar10.load_data()\n",
    "Y_train = np_utils.to_categorical(y_train, nb_classes)\n",
    "Y_test = np_utils.to_categorical(y_test, nb_classes)\n",
    "X_train = X_train.astype(\"float32\")\n",
    "X_test = X_test.astype(\"float32\")\n",
    "X_train /= 255\n",
    "X_test /= 255"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f9601e40b38>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHz5JREFUeJztnVuMXNd1pv9Vt67q7uob2d0km5Qo6jaSbYmSGUGQPRl7\njASKEcT2iyZ+CPRghHnIGGMg8yB4gLHnzRmMHfhhYIAeK1EGHsdGbMdCIExgCw6ExIZHlCXrHomi\nKPPSbDbZ3eyqruq6rnnoEkK1979ZItnVlPb/AQSr96p9zj67zjqnzv5rrWXuDiFEemS2ewBCiO1B\nzi9Eosj5hUgUOb8QiSLnFyJR5PxCJIqcX4hEkfMLkShyfiESJXc1nc3sQQBfB5AF8L/c/Sux9+fz\neR8qFoO2TqdD+2UQ/hVi1vi+Cjl+XctHbLlsltrMwjs0i1xDI2Nst/kxx353mY2Nkfxis+tdvq8u\n35tlIgcQodsNH1ts7NHtRcZvkUlmtkxkHNkM/zzZOQAA3civZT12IrA+0e2FWVqpoFpb72tnV+z8\nZpYF8D8B/A6AUwCeNrPH3f1l1meoWMTBez8ctK2sLNF9DWXCH/xUgU/ODTuGqW16aoTadk6MUlsh\nmw+254ZKtA+yfIqXlleordnmxzY5MU5tmU4r2N5oNGif9fV1aiuWwhdrAOiAX7xq9WqwfXxijPaB\n8+01G01qyyL8uQD8YlMe5Z/zyAg/P/J5Ph/1yBg9doPIhM+R2DG3Pezff/6t7/P9bN5t3+/8Te4D\ncMzdj7t7E8DfAPjUVWxPCDFArsb55wCcvOTvU702IcR7gKt65u8HMzsM4DAADA0NbfXuhBB9cjV3\n/tMA9l3y995e2ztw9yPufsjdD+Xy/NlMCDFYrsb5nwZwq5ndZGYFAH8I4PFrMywhxFZzxV/73b1t\nZv8RwD9gQ+p71N1fivVZX1/HSy+H37Jy/jztN0UWWG0HX3nd2SlTm5VmqG2ty1WHaie8Au9WoH1q\n63zFtlbnK/CtDpe2zkc0zmIuPMZ2m28vS1abgfijWm19jdra3fBx2/oO2icTUQFbEbWilOPnQZWs\nmC912rTP8DBf7bcM//ZqRA0CAETkw9p6WKFpt8LtAJDNhT+X1nqdj2ETV/XM7+5PAHjiarYhhNge\n9As/IRJFzi9Eosj5hUgUOb8QiSLnFyJRtvwXfpeSAVDKEZkq8uO/G4mkt3+WB7jMTE9RWykm5USi\ntuqNcADMeovLUB7ZXqEUCQiKBPZ4l+9vfCoc0NRu8e0V8nwckWBLZAv8Q2s0w3PVavP5GI5sLzfC\nx1iM9GtbWI7MRKIE25EIvFgk6egIDyarrtWordUOS3qxgMrK6sVgezf2gW3eft/vFEK8r5DzC5Eo\ncn4hEkXOL0SiyPmFSJSBrvabOYoWDqgol/lQbpubDLbvKPFIkHyXp6aqLvFgm06XXw/rtfDYMzyu\nB2ORtGC5yCr1ysUK7xf51KbK4RXnyioPwmlGAnTqJOgEiOelGyWpsFpNHniS6fADy0cCjDokdRkA\n5MjyfKPB+xTy/APNdHlAUKO6TG0gQWEAMERO43aXKxIX18KKTyeSj3EzuvMLkShyfiESRc4vRKLI\n+YVIFDm/EIki5xciUQYq9eXMMDkU3mUpIuWMk6CO6TGeM61DykUBiNSZAbK5SCI5koet0Y1ITRFd\nLhcJLuk0uCTmWX7NPncuXAWo0+JHXanxoJNah8uio6VI9Z0GKdcFfswZ4zJVdihSKWeNy7rD+fAY\nc5FSWOuRvIv1Fpf6upEiaytVPsaVWvj8qRJpGQDWW+FzoBnJ1bgZ3fmFSBQ5vxCJIucXIlHk/EIk\nipxfiESR8wuRKFcl9ZnZCQAVbKhnbXc/FN1Z1jA9EZZsynkusRWLYVsmy6WVUiQ/XqvNZa9uJFLN\nPSwBNSP59jpNLgN2PRIxF5HYPMejzirNcIRep8PntxYpDdaO2CprfPynl8LjyGf49saqfO5bZ3k5\nt/pFLlXesPOWYPvMzF7ax8rh/HgA0Fi+QG3VKo+OvFjhUt/5i2FZ98RJPo5ONuy6jSaXBzdzLXT+\nj7s7/2SEENcl+tovRKJcrfM7gJ+Y2TNmdvhaDEgIMRiu9mv/R939tJnNAPixmb3q7k9d+obeReEw\nABQjz/VCiMFyVXd+dz/d+/8cgB8CuC/wniPufsjdDxVyesoQ4nrhir3RzEbMrPz2awC/C+DFazUw\nIcTWcjVf+2cB/LBX3ioH4P+4+/+NdcjnstgzHU7sOFbgEsXocFjasohUhkiElUWi6Rp1LhtliAy4\no8zLho2M8Gi01YtcJBkf4xFzlUhSzbdOh7dZbfBHrkIkEGxuOBKVmOeRhycuhKMLGx5JuhqJ6hsf\nK1PbA3dyhXl1Pizrei2yr508WrRR4/NRrfJ76VCeb3PfrvCxzczM0j4Lq2Hp8MJrZ2mfzVyx87v7\ncQB3X2l/IcT2oodwIRJFzi9Eosj5hUgUOb8QiSLnFyJRBpvAM2uYKoej7XLNsDQEAEP58DCHh8J1\n6QCgUedyWCtSb21iIlwXEACcJH1sdvg1tNWKJJcc5XX8ziyGa7EBwBtv8WivxUr42CK5IHFjpObh\np//tQWrbu5uP/2+fOR5s//kxLkW1uzySMZfh0lxlZZHaatXwPJbLXHpDh0cXFou8X4FEnwLAsPF+\n7U74w7lh3x7ap7wUruX4/Jt8LjajO78QiSLnFyJR5PxCJIqcX4hEkfMLkSiDXe3P5TAztSNoqy/x\nVfGMhYdZJWWOAKAeyWWWs0g+u0hZK3alrLf4KvXEJA/QaXb4CvbxU2eobWmVj5Hl98tGSnyNFfn2\nZnLhVWUAKC5xReLWsV3B9vkpPo6FlXPU1qjxOX72tdeoLUPKV7VGIqXGxnlADTLcZcbHufpU7kbK\ng5E8j95cpX32kwC5oXz/93Pd+YVIFDm/EIki5xciUeT8QiSKnF+IRJHzC5EoA5b68pjcOR20TY7y\n8lqZTDgoYmV1mfZprVX59jqxcl08oZ2TAKPRUZ6nrwVue+U4l6jWGrz0U7E4xG2F8BhLI1yGmsxy\nWfSZYwvU1m7y06cxHpb6pif5fBi4/NZqcym41uS5BNdIrr5mmx+zRaTbSDU35DORUm+ZSO7CXHge\n2w0upTqRiUnsWRDd+YVIFDm/EIki5xciUeT8QiSKnF+IRJHzC5Eol5X6zOxRAL8P4Jy7f7DXNgXg\nuwD2AzgB4CF357rbv24NILKdRcoZMYYi+dSGEY56AoBc5JqXyUTy8REZcKjEy3WdP8uj4mrn+ZQd\nmOKSWIOrXigSSe/2m+don0xkg+0sn+PViNSay4bzDJYL/HPZMXkztd186w3U9uavn6a2V187HWwv\n5CIymnOZuN3mLpMhEZUAkC/weex2w+dVN6IrmoXP04gS+Rv0c+f/KwAPbmp7BMCT7n4rgCd7fwsh\n3kNc1vnd/SkAS5uaPwXgsd7rxwB8+hqPSwixxVzpM/+su8/3Xp/FRsVeIcR7iKte8PONZPb0R4Vm\ndtjMjprZ0Uot8rAqhBgoV+r8C2a2GwB6/9P8S+5+xN0Pufuh8jBfxBJCDJYrdf7HATzce/0wgB9d\nm+EIIQZFP1LfdwB8DMBOMzsF4EsAvgLge2b2OQBvAXion5113VFfDycrtBaPzALCEVhrazzBYbPF\nr2vtDP8GUq1xaW6V2Ob28Wn0Nt/ejTu5MHPzHi4N1dZ5v7nb7g62F5w/ci1f5IlQSxPhhKsAgAs8\nUm3frt3B9pU1Hq144N/cSm1jkzwqcWzyDmpbXgzP//JFXvIsH5EjM84jKlvdSLQoDxZFpxU+vyNB\ngrR03LsI6ru887v7Z4npE+9iP0KI6wz9wk+IRJHzC5Eocn4hEkXOL0SiyPmFSJSBJvB0ODoWlkO8\nwxMqMlmjVORJP0fLXBo6s8hlxTdPLVJbLh8eR2GB19VbX+Dbu3WGy3mf+BiXvd44vTnU4l8pz4UT\npO7cEU6oCQDnFnmSzomJiOzV5eMvkISV5xbDUXYAkCuuUNviyjy1nZ7nUXj5fPg8mBjj2lu9zgUz\nz/H7pUW0uW5EBsxYuJ9FIkwjZR77Rnd+IRJFzi9Eosj5hUgUOb8QiSLnFyJR5PxCJMpApb5sNoOJ\nidGgrZ3jUl+1Go5I8xaXTy5WeNTWW7/m0la1ymWjUjF8rZx/k0cXzhZ5Use5uRupbWLPTdSWr0RC\nxEhS071338e7nOXyW6nNpcoOeKTg2lrYtns4LEUCQLPDj8tGwucNAOwd2UNt5YmwxFm5cJb2Obdw\ngdpaxuXN9SZPCooM1+ZGhsJRps16RMIkCUGNyIbBIfX9TiHE+wo5vxCJIucXIlHk/EIkipxfiEQZ\n6Gp/t9NGZSW8kppr8lx3eVKaCDyFHHJZbqxVuRIwWeaBLBMj4VXZ+jJf7Z/Zw3Pgzd3176jtxVNN\nanvtGLc9sHsq2L6ywvvM3hzO+wcAGdSordngSsCEh1fuV8/xlfRSk+cS3D0VPi4AWOnwvHr5uyaD\n7fVIoNA/P/E4tZ06yY85GynJFSukxeKIWrGycq3wXLEguOA2+n6nEOJ9hZxfiESR8wuRKHJ+IRJF\nzi9Eosj5hUiUfsp1PQrg9wGcc/cP9tq+DOCPAbyte3zR3Z/oZ4dZonh0IkEMTmSSDCnjBQAd41Lf\nMleUsLoayd/WCMtlu8e5PPhbH/84te29/X5q+8FfPkptuyJBLtlmOD/h6eNv8O0duJPaijtuobYR\n5/JsbSlcu7XUDUtvANCsc1nxfIXbJqZ5ENSOXfuD7fXqGO2T4SZ0CjyYKZbDr9XiUqu1wwFq5jxw\nrd0Ou+61lvr+CsCDgfa/cPeDvX99Ob4Q4vrhss7v7k8B4OlihRDvSa7mmf/zZva8mT1qZvy7nBDi\nuuRKnf8bAA4AOAhgHsBX2RvN7LCZHTWzo9Uaf+4RQgyWK3J+d19w9467dwF8EwBNE+PuR9z9kLsf\nGh3mWW2EEIPlipzfzHZf8udnALx4bYYjhBgU/Uh93wHwMQA7zewUgC8B+JiZHQTgAE4A+JN+dmYA\njCgRHRKlBPCyRZHKSfB6ZHuRFHhTO3iZr13DYWnx3kO30T53PMDlvOVzXN4cavPIwwN791Jblxzc\nrhmeO6+9ziXTWiQasNnm/Vr18KnVAZcp3zh9itpeePEotT1wPx/jjl3hqMrVSliKBABS4QsAsHM/\nl3W7sfJazYhsRyTki4u8fFmjEh5kl0RThris87v7ZwPN3+p7D0KI6xL9wk+IRJHzC5Eocn4hEkXO\nL0SiyPmFSJSBJvB0B7okgqne4BJFgUSx5XI8YWI2w+WfW3bxXyMXS/x6uP/GfcH2uz/KI/d2334X\ntT3387+kthv28THu+sCHqK0wfXOwPTc8TvvU1rnkWF/lkXsLZ05S2/JCWLbrtHh0XqkcTpAKADt3\n8s/65JlnqW1291ywvV2LRJHWedktW1umto6HIyoBwJnGDaA0FD62wi5+zKtDJNL1XXi07vxCJIqc\nX4hEkfMLkShyfiESRc4vRKLI+YVIlIFKfWaGfDa8y+VIgsbOeljWKA2XaJ9shksrM5HIvZPzPJLq\n5ntDqQyBvR8Kt2/AJbtWZY3axstcmpu+7SC1reXCNe1eevZp2qdR5+NYXeXzcf70r6kt2wlLrcUi\nP+XmbgrLcgBw1208kWg7yyPt8tmJcHuBR33m1nmSztpbp6mNydgA0I7cZqukruTwDn5cs6QGZD7f\n//1cd34hEkXOL0SiyPmFSBQ5vxCJIucXIlEGG9jT7aJRD6+kDg/xoVgxvBqaz/Acct7httIoL+X1\nB//hD6jtgd/7RLB9bOcs7bNw/BVqy0bGv1LhOfwWT/wLtZ2phFec//Hv/o72GS3xAJL1Bg+A2TXL\nFYmxcnil+s1TPBioGZmPqT37qe22D32Y2tAZCjYvrfB8gTWiLgHAcp2P0Zyfw+t1HrhWJSW2vMpV\nhzvCIga6/Vfr0p1fiFSR8wuRKHJ+IRJFzi9Eosj5hUgUOb8QidJPua59AP4awCw2ynMdcfevm9kU\ngO8C2I+Nkl0PuTtPcAbA4eg6ya3X5UER1g7LJG2PlOSK5EwrDo1R28EPc9loKB+WxF5+jueQWz7z\nBrU1GlzKqSwvUdvJYy9TW9XDwU75Dt/XaI5Ln2NFHlwyPcmlvvmFs8H2dqQsW63CZcWTb/IgIuAl\naqlWwzkIizl+frSHZqjtQpufO6USz0E4XOZBaKVcWI6s1FZpn3Y3LDm+C6Wvrzt/G8CfufudAO4H\n8KdmdieARwA86e63Aniy97cQ4j3CZZ3f3efd/Ze91xUArwCYA/ApAI/13vYYgE9v1SCFENeed/XM\nb2b7AdwD4BcAZt19vmc6i43HAiHEe4S+nd/MRgF8H8AX3P0dDyPu7iCPG2Z22MyOmtnRtTrPpS+E\nGCx9Ob+Z5bHh+N929x/0mhfMbHfPvhtAsOC5ux9x90PufmikVLgWYxZCXAMu6/xmZgC+BeAVd//a\nJabHATzce/0wgB9d++EJIbaKfqL6PgLgjwC8YGbP9dq+COArAL5nZp8D8BaAhy6/KQcQlu26bf5I\nkMuHc+51IjnTmuDRV7PjPK/ePzz+99Q2NRuWlGZ2h8t4AUCzxqPz8vmwxAMAoyNcUspluDQ3QuTI\nXTPhnG8AUK9whbaU5WO8sHie2lrN8GdTLnLJq1nlUt/rzx6ltvlXX6O2RpuU0MrzOezE5ncvlz4x\nws/hzBCXWotEtpsEn6s7PnBTsL1UPE77bOayzu/u/wSAxTiGY1yFENc9+oWfEIki5xciUeT8QiSK\nnF+IRJHzC5EoA03gCTd0u2HhoBCJLCvmSPLDDE+06JESTt0mjyw7fz4cjQYA1cWwrdTi0Vdd8OOa\nmuTy28SeaWprdxrUdvpMeIweiffKZPhp0GxzyTRrPPHnSDEsz5IAzY3txYyRKM1Ok8upGXK+rda4\nvNkcIvIggPIePvdrJV7arNLlMuD6WvgevGPsAO2zk0i3uXz/Lq07vxCJIucXIlHk/EIkipxfiESR\n8wuRKHJ+IRJlsFIfDBkLR4kVh3gEk5MIvZFSWE4CgJHyTmqrtXiE1Y4yzzmQI+NoXlygfboZvr1a\nnktbs7PhqC0A6Da5bHT7XXuD7T/76ZO0T9Nr1JY3LqfWq7zfWDkclVjI8VMua5F6duv8M3tznst2\nKyvhz6xha7TP9G38njg3EYlKdP5ZL5/nc1VYD0umI3ORSMxaOGqyG1FLN6M7vxCJIucXIlHk/EIk\nipxfiESR8wuRKANd7c8YUMiFrze1Bg+YyJKSUd1IfrlaiwdnZPM8SGSowFdz8/nwOArDvGzV+BgP\nMDq7yFWC2lx41R4AZvbdQm2nz4Xz6n3gtz5C+1QXz1Db8dd4Kay1Kg9kyWXD8z8+znMTGsnvCADz\np/kYf/1WJLBnKDz/Y7NcKZqeiowxojrYEv+sJ5e5q83NTAXb907wc+DYy+EArkadB61tRnd+IRJF\nzi9Eosj5hUgUOb8QiSLnFyJR5PxCJMplpT4z2wfgr7FRgtsBHHH3r5vZlwH8MYDF3lu/6O5PRHeW\nM8xOh683rQsXaL96JywBrfHYDHiGl/LKRYJLxsZ4MEWBlMKqr/EcfqVYTrUmtx392c+o7cDtXCI8\ndSosAWUi+Q6Hh3guvmxETi2VuLS1Vg1LffU6l2DbkZJtoyU+jgfuuY3aiiTAqJ3luQk7LR6EUz/J\npb5MpUhtM8Nlarvntg+E+0zwqvfPzL8ZbG+3+HFtph+dvw3gz9z9l2ZWBvCMmf24Z/sLd/8ffe9N\nCHHd0E+tvnkA873XFTN7BcDcVg9MCLG1vKtnfjPbD+AeAL/oNX3ezJ43s0fNjJe+FUJcd/Tt/GY2\nCuD7AL7g7qsAvgHgAICD2Phm8FXS77CZHTWzo6s1/kwnhBgsfTm/meWx4fjfdvcfAIC7L7h7x927\nAL4J4L5QX3c/4u6H3P3Q2DDPdCKEGCyXdX4zMwDfAvCKu3/tkvbdl7ztMwBevPbDE0JsFf2s9n8E\nwB8BeMHMnuu1fRHAZ83sIDbkvxMA/uRyGyoUDDfsC9/9x43LJMdOhqWXhUUendfscGlodJQf9lqN\nR4h1utVgezZyDV1a5BJmpcplmfUWH0fWua08Gl56WTi7RPucWuPyVde5RDg7zWVR64ajy5ZXeL69\noRH+mU2Mc6mskOXz32gSyTfH5c21Bt9esxopUdbl/W7Zt4va9uwKz+PJU1zSvbAY9ol2rOTZJvpZ\n7f8nAKEzIKrpCyGub/QLPyESRc4vRKLI+YVIFDm/EIki5xciUQaawDObM4xNksg4Il0AwORMNmwY\n4UkYzy/whKDrkXJXuQJP3si6dVs8grDV4eO4WOey10gkim29xqW5+no4gWczMsZOxOZO5h5AdTVS\nrmssnAh1bIwnO63X+fbOX+BzNTrKowstE76/WZvLxIUcT+I6xBVpFAp8rvbfsp/a6rXwWJ566mXa\n5/nXzoW3td5/VJ/u/EIkipxfiESR8wuRKHJ+IRJFzi9Eosj5hUiUgUp9ZoZcMbzL4hiP9Z8aDV+j\ncnUuo+VLPLppNVI3DR1+PSwVZ8Jd8nxfnQavZ1cY5uPI5/h8ZLNc4mx4eCzNFpc3PRK5Z1wRgze5\n5Nghpnwkmg4FLm+uLHOpr97k9enGJ8LSbY5IgACQicx9DVxKWzhfobblSARnZS0cpfmTf3yV74uo\noutNSX1CiMsg5xciUeT8QiSKnF+IRJHzC5Eocn4hEmWgUl+3a6iyBIjZUdpvdCSsG+VLXIcaiYRf\njY9zaa66ymvJVVfDCRWrtUhU3zq3lQs8AWaR1AUEgHaDS5y5XPh6Xohc5vNDPBrNjHccjiRCzRBT\nu8OlqEIpUkNxgsubS0tcYqsQ6XNsis99LVIz8PUTPCHrqy+cpLbZKR4tOruXHFuGn6c7SULThQqX\nPX9j832/UwjxvkLOL0SiyPmFSBQ5vxCJIucXIlEuu9pvZkUATwEY6r3/b939S2Y2BeC7APZjo1zX\nQ+7Ooy+wkQPv1FthW2OFr86Xp8MrxMVSJKCDiweYmuKHXV3jeeRWVsK25Qs8EGSZLw4j2+Wr7F3n\nSkanwxUEdMO22FXeMjywJ5vjc1WPBEE5WdTPkzJeANCu8ZJinUh+v04kWGilGu7HqngBwFJE8Tlx\njH+gKxfWqK25xne4azxcyuuOG+doHzbE18+u0j6b6efO3wDw7939bmyU437QzO4H8AiAJ939VgBP\n9v4WQrxHuKzz+wZvV6jM9/45gE8BeKzX/hiAT2/JCIUQW0Jfz/xmlu1V6D0H4Mfu/gsAs+4+33vL\nWQCzWzRGIcQW0Jfzu3vH3Q8C2AvgPjP74Ca7Y+PbwG9gZofN7KiZHb1Y5ckfhBCD5V2t9rv7CoCf\nAngQwIKZ7QaA3v/BKgLufsTdD7n7ofHRSMUDIcRAuazzm9m0mU30XpcA/A6AVwE8DuDh3tseBvCj\nrRqkEOLa009gz24Aj5lZFhsXi++5+9+b2c8BfM/MPgfgLQAPXW5Dbjl08juDtlbhEO3X6IYDWTLt\ncGkqACiOc/lqYpp/A5nM8MCTqVo40GJliZd3WjnP5bz6Gp/+TpvLh3B+ze62w2Ncr/NHrkIhki8w\nx8dfWeeBJ3XyiJd3HjRTzoSDVQCgm+ESVqvF53FoJCyZFvM8X+BEgY/xACao7UN387Jht991N7Xt\nv+WWYPt993N589SZarD9n9/gPrGZyzq/uz8P4J5A+wUAn+h7T0KI6wr9wk+IRJHzC5Eocn4hEkXO\nL0SiyPmFSBTzSPTYNd+Z2SI2ZEEA2Amgf11i69A43onG8U7ea+O40d2n+9ngQJ3/HTs2O+ruXNzX\nODQOjWNLx6Gv/UIkipxfiETZTuc/so37vhSN451oHO/kfTuObXvmF0JsL/raL0SibIvzm9mDZvYv\nZnbMzLYt95+ZnTCzF8zsOTM7OsD9Pmpm58zsxUvapszsx2b2eu//yW0ax5fN7HRvTp4zs08OYBz7\nzOynZvaymb1kZv+p1z7QOYmMY6BzYmZFM/t/Zvar3jj+W6/92s6Huw/0H4AsgDcAHABQAPArAHcO\nehy9sZwAsHMb9vvbAO4F8OIlbf8dwCO9148A+PNtGseXAfznAc/HbgD39l6XAbwG4M5Bz0lkHAOd\nEwAGYLT3Og/gFwDuv9bzsR13/vsAHHP34+7eBPA32EgGmgzu/hSAzXmqB54QlYxj4Lj7vLv/sve6\nAuAVAHMY8JxExjFQfIMtT5q7Hc4/B+DScqansA0T3MMB/MTMnjGzw9s0hre5nhKift7Mnu89Fmz5\n48elmNl+bOSP2NYksZvGAQx4TgaRNDf1Bb+P+kZi0t8D8Kdm9tvbPSAgnhB1AHwDG49kBwHMA/jq\noHZsZqMAvg/gC+7+jtQ9g5yTwDgGPid+FUlz+2U7nP80gH2X/L231zZw3P107/9zAH6IjUeS7aKv\nhKhbjbsv9E68LoBvYkBzYmZ5bDjct939B73mgc9JaBzbNSe9fb/rpLn9sh3O/zSAW83sJjMrAPhD\nbCQDHShmNmJm5bdfA/hdAC/Ge20p10VC1LdPrh6fwQDmxMwMwLcAvOLuX7vENNA5YeMY9JwMLGnu\noFYwN61mfhIbK6lvAPgv2zSGA9hQGn4F4KVBjgPAd7Dx9bGFjTWPzwHYgY2yZ68D+AmAqW0ax/8G\n8AKA53sn2+4BjOOj2PgK+zyA53r/PjnoOYmMY6BzAuAuAM/29vcigP/aa7+m86Ff+AmRKKkv+AmR\nLHJ+IRJFzi9Eosj5hUgUOb8QiSLnFyJR5PxCJIqcX4hE+f+zWYFHOK31HAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f960ae8b588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X_train[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f9601d78ac8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH/dJREFUeJztnVuQnWeVnt+1T30+t7rVklpqSZaEZNmWjVBs7BgSD9gQ\nUoaaxAUXE19Q47kgJFQmFy6mKpA7kgpMcZFQZYJrzIRwqAEGl2EyMcaDYXxCPulg2bKs86FbUkut\n3Yd93isXvV0ly9/7dcuSdsv536dKpe7v7W//X//7X/vv/b17rWXuDiFE8kgt9QKEEEuDgl+IhKLg\nFyKhKPiFSCgKfiESioJfiISi4BcioSj4hUgoCn4hEkrmSiab2X0Avg0gDeB/uvs3Yj/f1d3jA0PD\nQa1cnKPzquVicNzd6JxsrpVquRaupbM5qqVS4eMVCzN0TrlUoJrXalQz8N8tlU7zeanw63lHZxed\n0xI5H16rUq1Q4M8ZEP7kaN3rdEaxwM9VLbKO2KdUmVSt8nXU67HH4/MyGR5OmQx/zhzh6yD24ds6\nWUZhroBSqcwvnovXtJgfCmFmaQD/HcAnABwH8Acze9zdX2dzBoaG8Rff+h9B7fgbL9FjnTm0Lzhe\nq/HlD6/+ENVWr99Mtb7lq6nW2hY+3v69z9I5Rw7solplmr9opCO/W3dfD9Uyre3B8R133k3n3LCR\nn6vihXNU27vnFarV6+XgeLkSfiEHgNf37qZafuos1UrlEtUq5XDQnZvkL1wzc3yN1Ro/1rJl/VTr\n6++kWs2nw8eq0CkoFsKvDP/w9PN80iVcyZ/9OwAccPeD7l4G8CMA91/B4wkhmsiVBP9KAMcu+v54\nY0wI8QHgmm/4mdlDZrbTzHZO5y9c68MJIRbJlQT/CQCjF32/qjH2Ltz9EXff7u7bu7r5e1UhRHO5\nkuD/A4ANZrbWzHIAPg/g8auzLCHEteZ97/a7e9XM/i2Av8e81feou++NzanVasifD+8eD/TynVJf\nFrYHPdNN54ysXsfXUefbqKk63wWuz4XtpuL5STrHC3zneOXgENVWj95AtdEb1lBtxcpVwfEhYrEC\nQDbbQrVqb9g9AIDRVcv5vGp4t79Y5Hbe1Hnufpw9y12HTMTWhYV3+/sG+O/c2sHXeCF/nmotrTyc\n6s6tymwmvJb8hSk6p1wK7/Y78wADXJHP7+6/AvCrK3kMIcTSoE/4CZFQFPxCJBQFvxAJRcEvREJR\n8AuRUK5ot/+ycQcqYZutXOL229xc2DYa28g/TTwzO0u1WHJJ/2AkaSYbfq3csGEjnfPR27dTbeVw\n2JYDgJ6eZVSrZHg2YHtr2DbKRDLErBrJ3Jvl9luJPJcA0N4Wtgj7erm9uX7dFqrt2/cm1WB8HaVS\n2Lrt6e6jcyKJnbiQn6CaI3ydAvFMwfPnw9dqYY4nEbGMv8vpw6E7vxAJRcEvREJR8AuRUBT8QiQU\nBb8QCaWpu/1er6NKEjusynewW3JtwfELZ3lpp4HlfCd99Y08aWZodAXVsmwbOFJvqVLlzsIbp3hC\n0NzBM/wxU3xX+c3drwXHP7KZ76TfveMjVIvtHucj9RmOHjkZHM9lI7UVczxRa3AZd3aOHnuLPyYp\nazZT4G5QPs+vq0yWl8fr7uZJULF6h6w8YazOYEtL+Fq0RVXvm0d3fiESioJfiISi4BcioSj4hUgo\nCn4hEoqCX4iE0nSrrzQXtlg627gF1N0fTnK57ZZtdM7oug1Um44ksrx58BjV8nNhu2Zmitdam5zi\ndt6pcV4PrjuS2IMUT/h44sc/DY5nH+Cv8x+74y6qZbPcxly+nNui8LBdNnU+3J0GAF5+hXc3ykTq\nDHZ0cYuwWgtbleUZ/pylI7fEWFeeWo1bsJPnuH2YQtgijLX/6u0NJ6ClI23B3ntcIUQiUfALkVAU\n/EIkFAW/EAlFwS9EQlHwC5FQrsjqM7PDAKYB1ABU3Z0XrANgKUNLSzaoVdJddF6hrTM4fijP2yq9\n+vsXqXZuktelO3GS12jLpsMpU9kUz74qkbZVAFAscm1kGX9qTo8foVo3yfaansrTOfsPHeLrGBmk\nWjbL1zgyGm7ltYKMA8DRcW6zvrmba0Mj3BY9fJRYbBX+nNXLXKtF6ie25rgd2ZIJX/cAUCiGH7O7\nm1uYGdLiyy7jfn41fP5/5k5MXSHEdYv+7BcioVxp8DuAX5vZS2b20NVYkBCiOVzpn/13ufsJMxsC\n8KSZveHuz1z8A40XhYcAoLePfzRSCNFcrujO7+4nGv+fBvBzADsCP/OIu2939+0dneGNOyFE83nf\nwW9mHWbW9c7XAD4JYM/VWpgQ4tpyJX/2DwP4uc1XDMwA+N/u/n9iE1KpDNrbh4Pa6SmeaXfgWNjm\neX0vf61JRWyoWqQ1WGGaF3ZME0uvUOI22tQ016YjrbAOH99HtY42botuWr8pLEQsx3/83T9Qbc3a\ntVTbuIm3KRsYCGedtbTy56Wnm1tlqSovFjpb4vcw1vKqMMWzC2s1XnS1tY1bdjN5/pjdkczDltZw\nJl65HGthF84wrde5TXkp7zv43f0ggFve73whxNIiq0+IhKLgFyKhKPiFSCgKfiESioJfiITS1AKe\n6XQGvf3hLLEDx/bTeacOh7PO2rO8kOWFWV4ccyZ/mmoWsUqmpsPW3FSBW0MZksUIAIPDQ1Rr6wpb\nZQCwcoybLKPENjr02nN0Ttq4DVip8Sy2M2d5cdKbbtocHL9hwzo6ZzSSndd5+61U2/XGUaqViuHC\nsKVsJKsP3JarO7ekx8fD/QkBINfCbcyePnYdcNu5UAhntNZ98Vaf7vxCJBQFvxAJRcEvREJR8AuR\nUBT8QiSUpu72l0qzePvtcG29N94+QOedPPV2cLwWScLp6umg2qYNY1Tbunkr1U6dCe+wHjnD17Fs\neTiRCQDWrOdJM10D3AmYOM+P52fDzsjRI3xH/EykpdjmLVTCJzaGd/QBYHaG7EZz8wBe5q7D3ue5\nW7FhE2/bNryyNzj+/IvPBMcBYHyCJ2NVKny3v1jg6z8faVPW1hleY2znfpa0vbucxB7d+YVIKAp+\nIRKKgl+IhKLgFyKhKPiFSCgKfiESSlOtvtmZPJ5/5snwQoZJ7TkA6zffFBxvi7RV2rxlA9U2bVxF\ntVoxnBgDAJ4K21ez4A2LMtlwYgkApNNhiwcAKlWeCDI7fY5qPeWwFVWtOZ1z9DRPgmrtPMGP1d1H\ntXXrx4LjHrnfFKbCdekA4I0XXqWaF/h1sPXe+4LjN93ME4wKO7nV9/aBw1Rrb+fVqXt6B6g23+3u\nveTz/HkplcLnymX1CSEWQsEvREJR8AuRUBT8QiQUBb8QCUXBL0RCWdDqM7NHAXwGwGl339oY6wfw\nYwBjAA4DeMDduS/RoFKu4vSxsC126y3/gs5raQnXduvnrhxGVvA6bOcirZqOHeA2Wrkett9SxlPV\n0hluvdSc1yBENdZuLGw5AoDXwsfr7AnXTgSAyRmeJZjK8ezIunP7cL57e2gSn9HZyp+zsRWjVGtN\n83WkEK67eNNWnlHZ28st2McL/5dq46d4CKwcWkG1moVrQGYjLefy+bAduS8bbm0XYjF3/r8CcKlZ\n+jCAp9x9A4CnGt8LIT5ALBj87v4MgEtvh/cDeKzx9WMAPnuV1yWEuMa83/f8w+5+qvH1OOY79goh\nPkBc8cd73d3NjL7pMrOHADwEANksr2EvhGgu7/fOP2FmIwDQ+J92wXD3R9x9u7tvz2SamkoghIjw\nfoP/cQAPNr5+EMAvrs5yhBDNYjFW3w8BfBzAoJkdB/A1AN8A8BMz+yKAIwAeWMzBUqkM2jv7g1o2\n4hpNTYX/sGjp55bMXJV7SkXeXQttfV1Ua6kbeUBu9XnkDBcrPIuttY1PTEXaa9VT4XmdA9xqyjm3\nN9NtPHPPc9xrrVv4d7Matw5Taf47ZztyVGvr5Fq1FLZ1J09M0DkDHbxt2P2fvpdqO187TLWZSHHP\nYulMcLxEWnIBQG9X+NrPpCP+96U/u9APuPsXiHTPoo8ihLju0Cf8hEgoCn4hEoqCX4iEouAXIqEo\n+IVIKE391E0u14KR1eFsKkvx16FiMZzBNJHny8/18iy2SpVbQxb5FGJhJpwhVnG+9kyGF+KsprnW\n3s0z3IYGpqjm58L2UDnSY87qfP1tbW1US0VcpbqHj1ercVs0lY0UT03zNc7M8ixNIwUtWyLXW/4M\ntwHb2sNWNQDcfcfNVHvz7SNU2/P6eHB8Js+zLXOkMGy9Hsu0fDe68wuRUBT8QiQUBb8QCUXBL0RC\nUfALkVAU/EIklKZafW6AW9jOqUSsqLnpsJXTErGhpvORQpxFXjhzLs9toyxJ6uvq4Jbdsj5uDXX3\n8wy3Zb38d6tleqhWaAmfx3NreFZfqXaKaohkHtaqkexCkgFZS/FsS4tYfb39PLuwXouskVxXPT38\n/OZ4bRpMTUds1krYCgaAbZuXU623K3z9PPEELxZ6ZiJcCLcaiaNL0Z1fiISi4BcioSj4hUgoCn4h\nEoqCX4iE0txyuu4A2SHO1PnOcU84hwGjPWT7HcCH1vH6fp2tfKc3bfz1cDYf3uktzl2gc9o6KlTb\ntIE7AaNrVlEtlV1DtZmp8BpHR0b4Og7R4svo7icnH0B/H08+ymTCyVOxvBOPJAq1drRTrVrkO9wp\ncrxsLJEM3A0aGOyk2swcdx1mp8LJOwCwclm4ZuBn/+Un6Zy//eWvg+OZzOJr+OnOL0RCUfALkVAU\n/EIkFAW/EAlFwS9EQlHwC5FQFtOu61EAnwFw2t23Nsa+DuBPAbzTZ+ir7v6rhR6rq6MdH7vjw0Ft\n3ZZb6LyTJ04Ex1eu4FbZxg3rqbZ82RDV0s7tw2mS1FGKJL9Yij9eZwdP7Ons5BZbOsetyiyxTAuz\n4ZZQAHDbVm4djm0co1qlzm1MJ/eVap3bcp7m5yqd5Zdqpcj9wzpJdEll+H3PWvk6EJlXqvDzkUnz\n2pC1cvi6WhaxFe/6px8Jjj/34m4651IWc+f/KwD3Bcb/0t23Nf4tGPhCiOuLBYPf3Z8BwPNjhRAf\nSK7kPf+XzWyXmT1qZjzZWghxXfJ+g/87ANYB2AbgFIBvsh80s4fMbKeZ7ZyZ5cUOhBDN5X0Fv7tP\nuHvN3esAvgtgR+RnH3H37e6+vbODb2AIIZrL+wp+M7s4S+RzAPZcneUIIZrFYqy+HwL4OIBBMzsO\n4GsAPm5m2wA4gMMA/mwxB2tvb8OHb/5QULvxVm71FbaGbbuOHp5VxivFAW7cyklFLJn+jnAdtki3\nruira520kgIWqMUWsZRKpXC7rvU3rKZz2nLccizM8oxFT0UuHwtrHqmPV3eu1SLPWaxFVbkQPh+1\nOv+dU5nI9RF5RqcnueV75NAxqt15163B8bkKryfZTuzIiLP8HhYMfnf/QmD4e4s/hBDiekSf8BMi\noSj4hUgoCn4hEoqCX4iEouAXIqE0tYBnKpVCG8lk62zlLa862skyI8UKY4UiLWb1xSwlD1tz9Qq3\n7GL2lUWKSFYjZmXMznFSgLSzl2dAVmv8WLV6pCAkackFAI5acDwVW3yNa7UMt2AdkSebFIy1enh9\nANAS+Z2zNf6cdRT5PJ8IW44AcObgRHB81SZexPVsKvxp2cux+nTnFyKhKPiFSCgKfiESioJfiISi\n4BcioSj4hUgoTbX60uk0unrClpNHsunmSmG7xku8p1qJzAGA2ZlZqpUrfF6pFM6mq1a5VVaJZOBV\nIseai/R9m5vl2V5VkinY1d9D53T18L6GvV2DVGvNhfvxAUCN9V60SF89cK2rixc0nTzNz2OxELbE\n6nVefMrAf696jV9z3V3crl6zephqhbnw9eiRYqc9XWHLPB2xjy9Fd34hEoqCX4iEouAXIqEo+IVI\nKAp+IRJKU3f7p6by+NvH/y6o1bK/o/POnw8nPsxcOEvnpCK5HjEnYGIifCwAqJFsof5I+6++wQGq\ntaT56Z89F27hBAD739pHtfxMeHd7dC1vyZXOcqelu4uvf+1aXhdw1Wi43uHadSvpnP4WnpXS1crX\nWI/UckQ6nGxTqfGd9HSkJVc6ssbhsYgz0s2dgIqHk4zS3HRAf3/4d85Ekt0uRXd+IRKKgl+IhKLg\nFyKhKPiFSCgKfiESioJfiISymHZdowC+D2AY8+25HnH3b5tZP4AfAxjDfMuuB9z9fOyx8tMzePLp\nZ4Na76pNdJ7XwvbVK88+TeesWcXrnw0OcPvqxPFxqlVJ3bf2fp4YU07xpJ+J47yF0z077qDatptv\npNpcqRgcT2X5U33o6BGq7X/rbart3vMK1Xp7wk1Z//hffY7OufPGjVTLRXqirRoZpVqZWH0WKXYX\nq7tYIbUJASCVidQF7OWJSW0kGaee5pY0Mz4jJSjfw2Lu/FUAf+7uWwDcDuBLZrYFwMMAnnL3DQCe\nanwvhPiAsGDwu/spd3+58fU0gH0AVgK4H8BjjR97DMBnr9UihRBXn8t6z29mYwBuBfACgGF3P9WQ\nxjH/tkAI8QFh0cFvZp0AfgrgK+6ev1hzdwfCxdPN7CEz22lmO8tlXghBCNFcFhX8ZpbFfOD/wN1/\n1hieMLORhj4C4HRorrs/4u7b3X17Lsc/3yyEaC4LBr/Nt7f5HoB97v6ti6THATzY+PpBAL+4+ssT\nQlwrFpPVdyeAPwGw28xebYx9FcA3APzEzL4I4AiABxZ6oL7+AfzrL/yboNYytIHOm5sO229v7X6N\nzhlZzu2fVKTOWVsrzxAr18MtlzZu5WvvG+EZf3ODvI7cZz71R1Rr72qj2iyx+iKdtVAlbcgAoFgN\nPx4AnD59jmpHDp0Mjre38/M7fnySaof3vkW1VJGv8eB48A9S7PjkdjpnzdgKqsWyAVOtkTS8LLcB\njdXqMz4nZ+Hn7HKsvgWD391/D4A95D2LP5QQ4npCn/ATIqEo+IVIKAp+IRKKgl+IhKLgFyKhNLWA\npxnQkgu/3ux/Yw+dl78Qtvo8ln1V5hlRM5F2XRbxSlpbwrlUlTnePuvCGb7GiaM8q+/v/j5c6BQA\nzk9HjjdzITje1c0ttp6+cAs1AOiIFJ48fjxs5wHA0GC4UGdrN7c+f/dL/jufe2sX1Wpl3hLtwHi4\nIOvxSMuzDZu5ddvT3c61Pt4Sra2dZ/X1dISvq2wrL8bZ3h5+XtwX7/Xpzi9EQlHwC5FQFPxCJBQF\nvxAJRcEvREJR8AuRUJpq9dWrFUxPhm273/zil3TesfHjwfFUJZxlBwC7duWpFkt9qlZ51hZIJtWT\nT/yGTslluVW27dbbqFbOdVEtX5qj2sGj4Sy2yUne369c5Fl9J8cPU+3QYf6Y22/9cHD8333pP9A5\nLz7/HNWqF3jGX77Ei8QUwjVmcHAnt1l/99IpqnVkuK2YzXFrLt3Cr4MuYvWtWjNG59z/x58Pjper\ni7+f684vREJR8AuRUBT8QiQUBb8QCUXBL0RCaepufzabw8jwSFDbMLaWznOEd6MzkVZY6ciOfirN\nX/O8zhNxcq0dYSHLkzZWrAgnuADAx++9l2pd7ZEEklZe++/1PeG6hvsP8LZby1eOUa0YaZOVbuNr\n3LP/jeD46/v30zntY5updvIk/537erk2lAvX1Wvv5HUQz43z9mWTJw5Q7czZcBIRABRrkSQ0UmDx\n1BQPz4/eE55T5WX/3oPu/EIkFAW/EAlFwS9EQlHwC5FQFPxCJBQFvxAJZUGrz8xGAXwf8y24HcAj\n7v5tM/s6gD8FcKbxo19191/FHqtareLcmXCLp9v/yUfpvI9+7GPB8ZYWnkiRidh5sXZd9UjrqjTC\nx6uUub9SKPMknMnjh6h2rsgTSM6d5W2yDhJL7+TpcEIVAHQO8fZUaOE2puW41VeuhpNtnvzt7+mc\nNetvotpoP7dMW1P8Mm4niVWlIq/hdzC/l2qdXbwWYs15Utj4+RmqDQ6OBcfnKvxa/M1vXwyOT0/z\n+pSXshifvwrgz939ZTPrAvCSmT3Z0P7S3f/boo8mhLhuWEyvvlMATjW+njazfQD4y7AQ4gPBZb3n\nN7MxALcCeKEx9GUz22Vmj5oZ/5iVEOK6Y9HBb2adAH4K4CvungfwHQDrAGzD/F8G3yTzHjKznWa2\nc3qGv88SQjSXRQW/mWUxH/g/cPefAYC7T7h7zd3rAL4LYEdorrs/4u7b3X17VyevTiOEaC4LBr/N\nt7D5HoB97v6ti8YvztD5HADeckcIcd2xmN3+OwH8CYDdZvZqY+yrAL5gZtswb/8dBvBnCz1QKmXo\nIG2GJvNFOu+VXS8Fx4eG+DbD8NAg1SoVbqOdPz9FNRTDa8zU+eOtXMtttNE+/pfQif28jtzsDK9Z\nNzS8PDjePtBL56RbuX01V+DPy8jIaqqNnwzXXTw7GW4nBgAjKyJt1CKt2WZK/PwjE77eKnVuz7a0\nkexNAC2RbNHy5BmqIRWu0wcAwySrslziLefY6eBn6b0sZrf/9wBCv3HU0xdCXN/oE35CJBQFvxAJ\nRcEvREJR8AuRUBT8QiSUphbwTBnQkg1nKpWK3GJ79tmnguNe4TZUdzsv0Fip8OyrYoG3AMuQ18o1\nY6N0ztbbt1Bt/WpuA04dC1tlADB+/izVcm1ha2v9QNgCBIAzZ3jG2U2btlLtxps2Ue1H/+v7wfEM\nwgU1AaAyy5/PcplrHqta2Rp+rmPts8bWrqPa6WNv8mOleJZpWwc/3ubNG4PjxTn+vIyODAXHf5vj\nluKl6M4vREJR8AuRUBT8QiQUBb8QCUXBL0RCUfALkVCaavXV63XMFUhBy0hRzXs/9Znw45V5Flg6\nYufVa7wwoqe5XZPOhG2q1g5eyHJ8iluH01O8b925Al+/tfKimm++ejA4Pvkczzhbt5Zbdh+5YQPV\nypGMv7Zc2NrySEZlLIMwleaXKml1BwAo1Emfxxo/v2tWcauvODNJtS3dPBvwxZdeodrJI2H7sDDL\nr2+fOx8cL5d4xuel6M4vREJR8AuRUBT8QiQUBb8QCUXBL0RCUfALkVCam9WXMnR0hu2ynkjlwa5l\n4aynUsTWaI28ruWMZ5Z5G88GbGkPz6sXefbV9HSeaul2XjhzaD0vuLm+nWf1vXUo3KsPxi3MLCmq\nCgAnTh2l2sAgL6DKtHKB21elEi/uORvJ+CtFst8qpbC1nGnl9uzwimVUO3JqgmoTR8m5B1Cc4b/b\n23tfDY4PDPB1eF9/eDxS6PRSdOcXIqEo+IVIKAp+IRKKgl+IhKLgFyKhLLjbb2atAJ4B0NL4+b9x\n96+ZWT+AHwMYw3y7rgfcPZxt0KBeL2JumiSz1PnrUNY6g+MTE3wH9a3XD1OtNcN39HM9fJd9kLQH\nWzHYQ+dkIglLAz0DVIvkHqFY4Kd5aCjsIKxcEd4dBoBT4+NU279/H9XGymupxpyY6Wn+nM3N8Z30\n/AXumsR2+2vlcGJVuoUn4ezdw1u9xVpoDQ0NU23lzbwW4tCy8LzBZbzuYitZ/1P/+DSdcymLufOX\nAPxzd78F8+247zOz2wE8DOApd98A4KnG90KIDwgLBr/P885La7bxzwHcD+CxxvhjAD57TVYohLgm\nLOo9v5mlGx16TwN40t1fADDs7u+0kh0HwP/mEUJcdywq+N295u7bAKwCsMPMtl6iO0h3YDN7yMx2\nmtnO6WlSyEMI0XQua7ff3acAPA3gPgATZjYCAI3/T5M5j7j7dnff3tXFP1IphGguCwa/mS0zs97G\n120APgHgDQCPA3iw8WMPAvjFtVqkEOLqs5jEnhEAj5lZGvMvFj9x9yfM7DkAPzGzLwI4AuCBBR+p\n7qiTtkupyOtQphJOSukmrb8A4KXnf0u18QmeGGNZnuSyY8eHg+N33bGdzrlwgVtbu15+gWqzRZ7I\nsv/oMaodPHw4OF6Y42+53HkRvNZunlySz09TbZq0FJvNc5syUooPmTRXeyJ/Ua5YG7Yj+wZG6Jyh\nFdxiW3HrTVTrj9Twy8VqQzItkowFD8dLKtIy7FIWDH533wXg1sD4JIB7Fn0kIcR1hT7hJ0RCUfAL\nkVAU/EIkFAW/EAlFwS9EQrHLqfl1xQczO4N5WxAABgFwz615aB3vRut4Nx+0daxxd+7PXkRTg/9d\nBzbb6e7cINc6tA6t45quQ3/2C5FQFPxCJJSlDP5HlvDYF6N1vBut4938f7uOJXvPL4RYWvRnvxAJ\nZUmC38zuM7M3zeyAmS1Z7T8zO2xmu83sVTPb2cTjPmpmp81sz0Vj/Wb2pJm91fif98K6tuv4upmd\naJyTV83s001Yx6iZPW1mr5vZXjP7943xpp6TyDqaek7MrNXMXjSz1xrr+M+N8at7Pty9qf8ApAG8\nDWAdgByA1wBsafY6Gms5DGBwCY57N4DbAOy5aOy/Ani48fXDAP7LEq3j6wD+Y5PPxwiA2xpfdwHY\nD2BLs89JZB1NPSeYz27ubHydBfACgNuv9vlYijv/DgAH3P2gu5cB/AjzxUATg7s/A+DcJcNNL4hK\n1tF03P2Uu7/c+HoawD4AK9HkcxJZR1Pxea550dylCP6VAC6uRnEcS3CCGziAX5vZS2b20BKt4R2u\np4KoXzazXY23Bdf87cfFmNkY5utHLGmR2EvWATT5nDSjaG7SN/zu8vnCpJ8C8CUzu3upFwTEC6I2\nge9g/i3ZNgCnAHyzWQc2s04APwXwFXd/V5eOZp6TwDqafk78CormLpalCP4TAEYv+n5VY6zpuPuJ\nxv+nAfwc829JlopFFUS91rj7ROPCqwP4Lpp0Tswsi/mA+4G7/6wx3PRzElrHUp2TxrEvu2juYlmK\n4P8DgA1mttbMcgA+j/lioE3FzDrMrOudrwF8EsCe+KxrynVREPWdi6vB59CEc2JmBuB7APa5+7cu\nkpp6Ttg6mn1OmlY0t1k7mJfsZn4a8zupbwP4iyVawzrMOw2vAdjbzHUA+CHm/3ysYH7P44sABjDf\n9uwtAL8G0L9E6/hrALsB7GpcbCNNWMddmP8TdheAVxv/Pt3scxJZR1PPCYCbAbzSON4eAP+pMX5V\nz4c+4SdEQkn6hp8QiUXBL0RCUfALkVAU/EIkFAW/EAlFwS9EQlHwC5FQFPxCJJT/ByGKsM3TKcRx\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f960ae8b2b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X_train[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To reduce the risk of overfitting, we also apply some image transformation, like rotations, shifts and flips. All these can be easily implemented using the Keras [Image Data Generator](http://keras.io/preprocessing/image/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Warning: The following cells may be computational Intensive...."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "\n",
    "generated_images = ImageDataGenerator(\n",
    "    featurewise_center=True,  # set input mean to 0 over the dataset\n",
    "    samplewise_center=False,  # set each sample mean to 0\n",
    "    featurewise_std_normalization=True,  # divide inputs by std of the dataset\n",
    "    samplewise_std_normalization=False,  # divide each input by its std\n",
    "    zca_whitening=False,  # apply ZCA whitening\n",
    "    rotation_range=0,  # randomly rotate images in the range (degrees, 0 to 180)\n",
    "    width_shift_range=0.2,  # randomly shift images horizontally (fraction of total width)\n",
    "    height_shift_range=0.2,  # randomly shift images vertically (fraction of total height)\n",
    "    horizontal_flip=True,  # randomly flip images\n",
    "    vertical_flip=False)  # randomly flip images\n",
    "\n",
    "generated_images.fit(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can start training. \n",
    "\n",
    "At each iteration, a batch of 500 images is requested to the `ImageDataGenerator` object, and then fed to the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(50000, 32, 32, 3)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "gen = generated_images.flow(X_train, Y_train, batch_size=500, shuffle=True)\n",
    "X_batch, Y_batch = next(gen)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(500, 32, 32, 3)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_batch.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f95d9b0e4a8>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH2FJREFUeJztnX14lOW17u9FJhAk0QkEIUIsUaiiRMEGJW6wgriLttZS\nra21frTuYutH61FPjx+n1Xbvdld3v7z2qXZTodVq8RsVK9aP4gG2VBMEDRo0aIIJTZBURhnNIBPW\n/iPjORifezKSMMH93r/r4iJ57nned80z78o786xZa5m7QwgRPQYNtAFCiIFBzi9ERJHzCxFR5PxC\nRBQ5vxARRc4vRESR8wsRUeT8QkQUOb8QESXWl8lmNgfAjQAKANzi7j/N9vjBBTHfJ1bYl1PmzJAh\nxVTrTHVSLVXFtYnvHhIcf3FoEZ0T37CBajvdsmj8m5dd6TTVxhwwNjheEOMv9esbW6lmWW4Pjveo\nFisI25/esZPOqRz3SX6unXzei0OHUK0w9nxwvGDNYDrnvSld3I4sX4gduo5f22b8tWYMGpRl8cnx\nOnek8F56R04ns939eq+ZFQB4BcCJAFoB1AI4091fYnPiQ4b69DEH7db5PioTxtVQrb6Rmoj1G16g\nWt2alcHxw6sOpnNOOeVUqqXS/I9GKp2iWqLj71T7lx/eEByPl46icy6+8DKqxYr4H400NlKtLL49\nON7Rzp/X7QufoFoq+Q7VDq/i11R52QHB8Xh8DJ3TkkhQLcvfXUwcz49ZlGUd6Zyhw7hI/pivalqD\ntzq35eT8fXnbfzSADe7+mru/B+BOAPxKF0LsVfTF+ccAaNnl99bMmBDiY0CfPvPngpnNAzAPAIYW\n5OfzvhCid/py598EoGKX38dmxj6Au89392p3rx5cUNCH0wkh+pO+OH8tgAlmVmlmgwF8BcBD/WOW\nEGJPs9tv+909bWYXA/gzukN9C939xX6zTAixR+nTZ353fwTAI/1kixAij+gbfkJEFDm/EBFFzi9E\nRJHzCxFR5PxCRJTdTuzZHYoLC31yfERezjWlaibVGlt5Yg9iPPGktnZtcDyV4okgFWN50klbB09W\nqamZSrUJE/gx6+vDz21D42t0TvVUfq54Kc+OTGEz1Urj4fGi9H50TtOzH/qO2P9j3cq/Um38+HDy\nDgCkY+HzJd7hGTop8ISrli38OU+deRjVqiYewY/ZEr5+0kk6Bcz6lU0rkOhM7PHEHiHExxg5vxAR\nRc4vRESR8wsRUeT8QkSUvO72D5p8hBc++ae8nOtLcy+gWkfiVarNnMF3ZZubwzu9q55elcUSXl+u\nqfVNqk0Yz3f0Z86aQbWykeFoSmsL30lPI1xyCwDa2vhaIcbnpUi9qymVU+icyeWHU62jgZdXGz2a\nR5CS6fD6tyey2J4l5aW+ga9HLM7LqxUX8zJqqVQ4IlG3ikdo1q0P23HMaZ1Yva5Lu/1CCI6cX4iI\nIucXIqLI+YWIKHJ+ISKKnF+IiLLHS3fvyqDnB6G4jCdN9CfpmTwroqyMz2vOkvSzbHldcHz0yHI6\np6aGJxjVz+AJNXWbXqda2+zjqZYmnX6K4yTTBgDAE5PatvCQ2Iqnaqn24H3LguPXXfNDOmfsDN75\naH0TD1UmOrn9sZJwGLBsLD9Xkud2Idn5FtXuveEKqiW28oN+5cIfB8fT4ElQiW3hpLAu3tXsQ+jO\nL0REkfMLEVHk/EJEFDm/EBFFzi9ERJHzCxFR+hTqM7NmANsAdAFIu3t1tscXDo6hfEx+avg9/sSf\nqVb5CT4vkeig2sOL7wmOH1MTru0HADUX/g+qJVM8HNncwkN91fXPUa24KBzHLCrk6z615kCqkcgh\nAOD2BQ9TbUVtODT36JJ6OmfO8dOpFh+Zpa5eO6+rV9QRDs1NKOKXfjLNz3XW18+j2vc+O4dqN/zq\nRqptWLsyOL5j7DQ6pyw5LDgeK+ykcz702JwfyZnp7txjhBB7JXrbL0RE6avzO4AnzGy1mc3rD4OE\nEPmhr2/7p7v7JjPbH8DjZrbe3Zfv+oDMH4V5AFBYwD9bCiHyS5/u/O6+KfP/GwAWAzg68Jj57l7t\n7tUFBSP7cjohRD+y285vZsPMrOT9nwH8I4B1/WWYEGLP0pe3/aMALDaz94/zR3d/NNsE9x1I7eBh\nmf7k3oceoFrFH3irpmOO48UxP9V5bHC8dCjPvlq1gme+NWcpqokkz1RrXNdItRdJ9ltZ+Rg656rv\nnEW1Dm4GqqZUUa28PNy6alh8XzpnQpJn2qWH8vBbWbY0TdLYKgFeUPO2v/DX7Ff/fhfVOq95gmrn\nX/JVqvnWl4PjI8u4ez4+PxzKfvvt8+mcnuy287v7awCO3N35QoiBRaE+ISKKnF+IiCLnFyKiyPmF\niChyfiEiSl4LeO6333qccnJNXs514y94SDHV8QbVqmLDqVYz6/Tg+JaGjXTO4lUNVGNhKAAYk+Bh\nwFfXrKFaPSkUWVzeRedsbXmFauk0zwacfRIPiyY6wnbUTOFz0ll63SW28tyx+g38tZ5SHS6gessi\nfq4XL+UR6x/U85Dj1y5aQrXZWa77jpfC/QTP+f536ZyZx4WPV1JSTOf0RHd+ISKKnF+IiCLnFyKi\nyPmFiChyfiEiSl53+21QDLGhPKGiPykv58k2ZRWfpVpx+blUu+ranwTHm+tfoHMSM3kdtliat34q\nzdJCa/YqXjNwfXu4LuCkCr7u357Ad/RLYwdRrbGBRyvKyHOrypJg9HgTb5XWlvob1ZKjeILRw7HD\ng+OzLuY19RY9/a9UK/71BVSr+tRjVOuI82hLUUW4Ht+PfsYjC0v/El6r5qZmOqcnuvMLEVHk/EJE\nFDm/EBFFzi9ERJHzCxFR5PxCRJS8hvrefTeNNc/lp7lPRxMPo02q5OGmJTfdS7XDDgwn1HTcfj2d\nUzX1k1QrSvKElHGjj6BaU1k4NAQAQ1aEk4xumH4hnXPDby6iWttEnoiz8jHeUmzpD38cHM+WdnLi\nyV+jWvUZy6l22U3nUe2w/X8QHN/Wwu34+rmnUa2o4VqqfarLqTZqHK8z+O3K2cHxdQ3v0DnbE+FQ\n8LL1vI5gT3TnFyKiyPmFiChyfiEiipxfiIgi5xciosj5hYgo5s7DEwBgZgsBfA7AG+4+KTM2HMBd\nAMYBaAZwhrtv7e1kw/YZ4hMPHdtHk3OjqYHXaCtMxal27VU3Ue2Vl1cHxy+5/Ct0zudP4CG7S77x\nBaotXbqCaqfN4SGxIoSf26B3dtA5c88Nt9YCgLNvWUC1eZ/ndnxxeDi78JTTvkfn3LCaBwIX1fIM\nt8GjeH28b88Lr/HsKTxb8eG/8FZvv/2fx1CtrJRnYj5d9xzV7r4vXPsvneLrsfnVcFbfMSdvx+rn\ndxqduAu53Pl/D2BOj7ErATzp7hMAPJn5XQjxMaJX53f35QDe7DF8KoBbMz/fCoDfwoQQeyW7+5l/\nlLu3ZX5uB7K0PBVC7JX0ecPPuzcN6MaBmc0zszozq0und/b1dEKIfmJ3nX+zmZUDQOZ/2gXD3ee7\ne7W7V8diCi4Isbewu974EID3i92dC+DB/jFHCJEves3qM7NFAI4HUGZmrQCuBfBTAHeb2fkANgI4\nI5eTOQzpPCUSJt7ZTrWNjfVUa36OZ9pddMk5wfH9fsmf0+SacMYWAHxm7uVUe23HLKpdeCNvC1U+\n+hPB8YUL/g+dc/Pyh6nWvIWHKm+67jdUm/mna4Lj5559Jp3zz9fzbLp/f4iH+vYr5VmO170bDt1+\ncwQv+vn6/w0XagWA1f/UM/D1/znr7G9SresNbv/tN98XHP/2PJ6JGYuFs1YNubfD69UT3Z29Wifk\nfBYhxF6HPoQLEVHk/EJEFDm/EBFFzi9ERJHzCxFR8lrA0wGkeXu3fqULKarVbQ0XuQSQdUW+emo4\ni22Z84ytt1rp959QOPQoqs1/5FGqbSiaSTX2BMb/SzicBACXnc8LeL66+ddUO5Zk7gHAWZeGC2cu\nqufZlvUH7U+1Wx+8k2pfeoQXrbzt2mXB8dQZ/Pr44+95qG/N6p9Tbd2KcIFXALi3gtu45qVwht4D\n9yykc8YfOjw43rqtk87pie78QkQUOb8QEUXOL0REkfMLEVHk/EJEFDm/EBGl1wKe/UlpaYmfMGtK\nXs7V0cKz8+qzhJv+voH3R+toIf3/Snh8cOljvADm0uU8RPhUG8/OWrKM249U2P5Dq3ixpeLYgVSr\nrDiEanOreN/FhgWnBMdrG3iY9eCvXU21u1/ga/XFz5xOtRFvh8NoUyvpFKTS/PVMJbNoG16mWmsT\n72vIgo7tCd5vsiX1SvhYx9aga/XqfivgKYT4b4icX4iIIucXIqLI+YWIKHJ+ISJKXhN70tvfQ0eW\nXc/+JJngu/2FiSFUG198ANVS28PaAw/x+qW3/u77VEuX8ZZRsaEzuB1p/ty2NYV3t8c1hWv7AQDK\neJJL/VZ+LjTyOonJpv3Cx1vGj3dFKjwHAGKdvAZeKsUv46LSMcHxcVX8XGtXraLaiuc2Ua3qgBFU\na07yCM2kieEIWGt7+LUEgGdWvRYc/3KSvyY90Z1fiIgi5xciosj5hYgocn4hIoqcX4iIIucXIqLk\n0q5rIYDPAXjD3Sdlxq4D8E0AWzIPu9rdH+ntWDt27EB7W5bQUT+yfl0D1ZYsfJZqNbVHU+3i48Pt\nuq5L8rZKM2ecSLVFj/Mkl4a3+TrVZinTFivqCo5vaOThq1SWEFt8JNeSnbwgY7o03OYrNvZYOufe\nJbyF1u8eXUy11AKePDV19tTgeFHsb3TOtKnhOQDwxS9fRrVZx/Nrp7kxHJoDgOKSsuB4WbKcH29k\nOFGrIFZI5/Qklzv/7wGEGpT90t0nZ/716vhCiL2LXp3f3ZcDeDMPtggh8khfPvNfYmYvmNlCMyvt\nN4uEEHlhd53/ZgAHAZgMoA0ALWZuZvPMrM7M6rp25q9wiBAiO7vl/O6+2d273H0ngN8CoDsd7j7f\n3avdvbpgUE4FRoQQeWC3nN/Mdt2GnAtgXf+YI4TIF7mE+hYBOB5AmZm1ArgWwPFmNhndHbiaAVyQ\ny8n23XdfzJzFs9X6ky+cchrVWpt4ZlYsxpdk7mnhsF1iGw/LNTVnCedlqSWYCndjAgCUlMSptqU9\nnKG3cMH1dM43av8X1SbO4y3Fiht5K7KOonAoqnwir0142ZXhFl8A8KN/DrdKA4ADR/H6fo/e/93g\n+Enn8JBjOsVfz9vvuYNqNoyHUxNbeagPCIcda47mNfyKisOtzawg9354vTq/u58ZGOaBVSHExwJ9\nw0+IiCLnFyKiyPmFiChyfiEiipxfiIiS1wKevnMndqRyLzDYFyoreHHMmcfxcOOSR5ZQrbHp1eB4\n3XM8xFP313qq/axjGtVKy3k4b11RgmrtbeEw5uKH7qVz5v+Gr9UhB6+m2oSRQ6lWFEsGx+NJXohz\n6YPcxg2NX6fa9Bk3UW3JQ+cGx7/zVR7qW7OGZ31uS/EwccdWHrotzhKeLSkKF/5saGykc1KpcEjv\n3U7euqwnuvMLEVHk/EJEFDm/EBFFzi9ERJHzCxFR5PxCRJS8hvq2Jd/FyuU8A6s/SXbw/nOTpxxC\ntXVrs2RfTQ4Pp8JRLQBAIsFDQ/HxPOxZXj6MHxQ8pBQvDYeNSkt4H7k5Z/FMu42Ny6h23hn/RLXK\n/x0Ol23lES8kUrVU68gSYrvqmsOoVlERzi5csYKH877zLZ6k+tSlPLtw7ud5sdYhhftSbVNLuEpe\n7fIX6Jw0woVaPR0eD6E7vxARRc4vRESR8wsRUeT8QkQUOb8QESWvu/2FhYUYTXZf+5tk7B2qdWzh\nkYCyEdy+ibPCUYKqKr7b/P0f8fZO24pmUm38BL47n0q9RLXLvhPeqb7sQp7MdPddvC5dZSW34/qf\nnEW1OceGd74f/csaOidb2CSxldemKy4uyDIvnAT1vSuvpnPeueIVfrwtvL7fpPEHUm1GKa+FOL0p\n3Kbs3nv+TOekSOJUYWHu93Pd+YWIKHJ+ISKKnF+IiCLnFyKiyPmFiChyfiEiSi7tuioA3AZgFLrb\nc8139xvNbDiAuwCMQ3fLrjPcfWu2Yw0qAIqK+2pybsQreIgqHRtCtbIUDwPG0uHWW7Hi/eiccZU8\n/NPRQSXssy+3PwYe9vrxd8MhtkW38VZYMz7PE1nqHvkPqhWV81qCJ36D9G6N88yemhqevNPYwi/V\nE2Y/TrXxFeGagTPP42HWpQ2/p9rosjFUKyvlyVit9TxpqaU13JaruI2HFeNDw9dcYSEPe/Yklzt/\nGsDl7n4YgGkALjKzwwBcCeBJd58A4MnM70KIjwm9Or+7t7n7c5mftwFoADAGwKkAbs087FYAX9hT\nRgoh+p+P9JnfzMYBmALgGQCj3L0tI7Wj+2OBEOJjQs7Ob2bFAO4DcKm7v72r5u6O7v2A0Lx5ZlZn\nZnXv7ci90IAQYs+Sk/ObWSG6Hf8Od78/M7zZzMozejmAYLN2d5/v7tXuXj34I2xGCCH2LL06v5kZ\ngAUAGtz9F7tIDwF4vx3KuQAe7H/zhBB7ilyy+v4BwNkA6s1sbWbsagA/BXC3mZ0PYCOAM3o70PbU\ndjQ3ZqmR14/Eh/JQ2diJvD1VUZYlSSEc6utM8I8zyfYDqBaPHUy1Ut4JC0VZMhZnHBF+3lP/9XI6\nZ0UHD1/deWP4OQPA9q7gmz0AwJTjVgTH46U89HlKim8b/dvPH6ZavIiHPlfVh1uprVzJQ2/lcX68\nIkzJYge3vz1LePY/l4fXKj2BXx/HVE8Kjr+1vZPO6Umvzu/uKwEYkU/I+UxCiL0KfcNPiIgi5xci\nosj5hYgocn4hIoqcX4iIktcCnjt3HoVkkrdJ6k+KYzzEtvhu/pWEqTXDuTY1nLXV2lTP7QAPKxbH\nj6faoOFZWnLFeebhfq+FMxZHjOQv9alX3EW1JfcvodpBB/+Nas2JcCuykXGeUTl+4lSq3XP/fKq9\nvuFFqq1avjQ4PngIL4I6vZq/ZtkKeMZIqzQAiBXxzM80cUPWeg0AqqqOCI6vfpG3+OqJ7vxCRBQ5\nvxARRc4vRESR8wsRUeT8QkQUOb8QESWvob6uyWlsfZqHsPqVOA/JvFjPwzzjVv2VapNJqC+Z4plv\nZQle1LGsMhwOA4BhI7L0hJvIs8cKRodDSps3NNA55yR4xtySux+g2mmnH061RDpc3HNcOe9rOCLO\nw2E/uyHczw4ABnXxurGtLeG+e2+98Tqds2F9uOgnABxfzV0m0cav7bq1fP1b28IFPIuK+LWzoT6c\nHbt9O7+meqI7vxARRc4vRESR8wsRUeT8QkQUOb8QESWvu/2DnitASZYEh/5k0lS+q3xMTQ3VHlhx\nJ9VuXxyuw7alhe8cT6nkS3z6ibxf16eP5H3NJk7kNffefDXcDmvJXXxHf+J4Xldv0hieINXRyhOM\ntnaGn/dns6z9pmZ+vGRyNtUGD+aJSTGEj3nisSfROcPS/DWrSmTZ7W/nu/1L5vOWYh2d4euq6Gme\npPOHheHktG9dcx6d0xPd+YWIKHJ+ISKKnF+IiCLnFyKiyPmFiChyfiEiSq+hPjOrAHAbultwO4D5\n7n6jmV0H4JsAtmQeerW7P5LtWEdUbcd/LslPu67qaZuotqmZJ8387hGupYvCbbla1/AQT0kNX+Kn\nlvJkj8t/zRNqDq38JNUeag+HD5d+moeaai74HtXOPGMu1W78t59QDbHw8y4Btz3ZtoZqp59zLtWu\nuoYnwBxaEa7HN2f2l+mcor+GW3wBQNN63qJsQnwi1WpWzaTaH+4Ph2GTHfy6Gl9B1tFzv5/nEudP\nA7jc3Z8zsxIAq83s/Svpl+7+s5zPJoTYa8ilV18bgLbMz9vMrAEA/5aJEOJjwUf6zG9m4wBMAfBM\nZugSM3vBzBaaWWk/2yaE2IPk7PxmVgzgPgCXuvvbAG4GcBCAyeh+Z/BzMm+emdWZWd2WN3nRBSFE\nfsnJ+c2sEN2Of4e73w8A7r7Z3bvcfSeA3wI4OjTX3ee7e7W7V48crjcHQuwt9Or8ZmYAFgBocPdf\n7DJevsvD5gJY1//mCSH2FLns9v8DgLMB1JvZ2szY1QDONLPJ6A7/NQO4oLcDvZNMoe5pXj+vP1m7\ntpFqiWQ4iwoAElv4knQkwrXWYkmeqVhWyjPmhsSy1Kwre5RqKOE1/B5DWXD87LNOp3NayfMCgJJY\nAdVqjppBtdqGcO28e5fwGompND/X1Jr/oFoi+Q7VyirCe9OtrTyku7mFa8cfzWsJTp16LNUuvpCH\nKm9fHA71pbbwenzxknBbueJh/NruSS67/SsBWEDKGtMXQuzd6Bt+QkQUOb8QEUXOL0REkfMLEVHk\n/EJElLwW8BwyZB+Mr5yal3ONivMQ28spnpmVbOfHHBcPZ1LFx/HwyvgJ4dAbAEyZNoVqX1rMC122\ntvEQ0BWtyeD4ibOC38ECANy8YCnVVqxYRrXLfnA11X41Pxyaa2jixU4/fQq3saqKF7PcmuQZnGef\nEw5Hrlz1FJ2z+P47qNZQUke1cUmeKVgcH0G1OdPCPtG8gYccd6TDmYxvFYZf/xC68wsRUeT8QkQU\nOb8QEUXOL0REkfMLEVHk/EJElLyG+l5a/xIm1/DwVr/SNYRK6SxPe9oMHmKLoyg4XlkRHgeAEaP3\nodoEHEG16TN4Mci6Wp6xGC8LP7dDR0+mc25Z+CeqnfOti6g2teYoqs05JbyOa+t4yO6GaxdQbdEf\nl1Bt/bO8KGw8Hu41uP9O3gvxqcW8eOrGpgTVzqMKUFbCi4yOJZmH7cU82xKpcEjPBnkWKz6I7vxC\nRBQ5vxARRc4vRESR8wsRUeT8QkQUOb8QESWvob7Bgwdj7Jj8NPtp2LGRamNGj6TaxqPeplqiLZxl\nlU7zkMzq2lqqHbKOh/pqanjI8ZCqcPFGAKgYGy7umSzmmYeTZ02n2k9H/o1q02bxDM1Vk8I98qZP\nPozOuekWnomZTvH1WPrYCqq1bYkHx8tKw/YBwOy5PDsvTnoQAsC4qfzaTnXyTMzy8vBrNq6SZ+it\nSoYLpHZ18XXqie78QkQUOb8QEUXOL0REkfMLEVHk/EJElF53+82sCMByAEMyj7/X3a81s+EA7gIw\nDt3tus5w96xteLe/l0ZTy9/7anNOTJjId9LTMb7zemQLT6h55vFwq6l0ij+ndLqLal1ZOit9Ls4T\nQWbO5M8NE8Mv6ZKneO289oZ3qda05WWqrVkwn2q1tWcExyeO5jvii267ndvRwpN32tt4tOWkz4Tb\nlC1ayM+FLK9LUypFtcbR3MbK8TySERsafq2Ly3g7t6LS8Jx0LPd2Xbnc+bcDmOXuR6K7HfccM5sG\n4EoAT7r7BABPZn4XQnxM6NX5vZv3A46FmX8O4FQAt2bGbwXwhT1ioRBij5DTZ34zK8h06H0DwOPu\n/gyAUe7elnlIOwDeOlYIsdeRk/O7e5e7TwYwFsDRZjaph+7ofjfwIcxsnpnVmVmdY2efDRZC9A8f\nabff3RMAlgGYA2CzmZUDQOb/YCcMd5/v7tXuXm0KLgix19CrN5rZSDOLZ34eCuBEAOsBPATg3MzD\nzgXw4J4yUgjR/+SS2FMO4FYzK0D3H4u73f1hM1sF4G4zOx/ARgDh2M4u7PwUkHy2T/bmzqG8rl5x\nnGvpFl6j7ZgZ04LjpcU8dLi1g7dcKi/jySUzjuP18YqLs7xsRJpy6Dt8yt0FVCvKdomkedjrmdp1\nwfFD11bSOeOz2NjcysNoY0bvT7X6hS8Fx19u4KFPkFqNAJDOEkkrTvJr58h2HkJuaiUt0br42ic7\nw4bs/AifrHt1fnd/AcCHqm66+98BnJD7qYQQexP6EC5ERJHzCxFR5PxCRBQ5vxARRc4vRESx7i/n\n5elkZlvQHRYEgDIAHXk7OUd2fBDZ8UE+bnZ8wt15kcpdyKvzf+DEZnXuXj0gJ5cdskN26G2/EFFF\nzi9ERBlI5+dlYPKL7PggsuOD/Le1Y8A+8wshBha97RciogyI85vZHDN72cw2mNmA1f4zs2Yzqzez\ntWZWl8fzLjSzN8xs3S5jw83scTNrzPxfOkB2XGdmmzJrstbMTs6DHRVmtszMXjKzF83su5nxvK5J\nFjvyuiZmVmRmz5rZ8xk7fpgZ79/1cPe8/gNQAOBVAAcBGAzgeQCH5duOjC3NAMoG4LzHATgKwLpd\nxm4AcGXm5ysBXD9AdlwH4Io8r0c5gKMyP5cAeAXAYflekyx25HVNABiA4szPhQCeATCtv9djIO78\nRwPY4O6vuft7AO5EdzHQyODuywG82WM47wVRiR15x93b3P25zM/bADQAGIM8r0kWO/KKd7PHi+YO\nhPOPAdCyy++tGIAFzuAAnjCz1WY2b4BseJ+9qSDqJWb2QuZjwR7/+LErZjYO3fUjBrRIbA87gDyv\nST6K5kZ9w2+6dxcmPQnARWZ23EAbBGQviJoHbkb3R7LJANoA/DxfJzazYgD3AbjU3T/QKz2faxKw\nI+9r4n0ompsrA+H8mwBU7PL72MxY3nH3TZn/3wCwGN0fSQaKnAqi7mncfXPmwtsJ4LfI05qYWSG6\nHe4Od78/M5z3NQnZMVBrkjn3Ry6amysD4fy1ACaYWaWZDQbwFXQXA80rZjbMzEre/xnAPwIIF57L\nD3tFQdT3L64Mc5GHNTEzA7AAQIO7/2IXKa9rwuzI95rkrWhuvnYwe+xmnozundRXAVwzQDYchO5I\nw/MAXsynHQAWofvt4w5073mcD2AEutueNQJ4AsDwAbLjDwDqAbyQudjK82DHdHS/hX0BwNrMv5Pz\nvSZZ7MjrmgA4AsCazPnWAfhBZrxf10Pf8BMiokR9w0+IyCLnFyKiyPmFiChyfiEiipxfiIgi5xci\nosj5hYgocn4hIsp/AerM/jv/7+YrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9601dc21d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X_batch[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f95d9a55cf8>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHSRJREFUeJzt3X10ldWVP/DvloAXvLGJJprURAMlVVyiwC9YsOIoHRlw\nrC+zpta+jTO14ozW6tJfO1Yr1elotZ1Sbcc6RUHxvdpqlY5aUGnBkVoiQYIEGyqhxCZAlKtEiBLY\n80euayJzvk8uebmBOd/PWi6Ss3Pu8+S5d3uTZ+fsY+4OEYnPAYN9AiIyOJT8IpFS8otESskvEikl\nv0iklPwikVLyi0RKyS8SKSW/SKQK+jLZzKYDuA3AEAB3ufvNSV9/yKGHeuVRR/blkDkrqFtJY21V\no2hsx/btNMb+GHLYgUPpnPTBB9HYusY/0Nj4neNobAWMxgqGrAiOH5Dwl5yHHlJMY0OH8u9t8+bN\nNJY+6CPB8SOP5M//bt9FY/UJ51HV1kpjm1rKguMOfj3aj32fxqrffpfGdry7g8be7+Cvq60fOyo4\nPvbddjpn587O4Hjzmxm81b6dv0C6sd7+ea+ZDQHwBwCnA2gGsBzA59x9DZtzwoTx/vRvf9ur4+2t\nkqJDaezueffT2Mq6VTS2c2f4xTmy+qN0zuSpJ9LYOWecTmOZljYaSyX8P7u0MB0cT3eGXywA8KXP\nn0NjZeVH0NiPb7udxk76xPTg+B0/+Qmd0975No2NLjucxubeeQuNzb75G8HxzoTrsbT+TzT29FO/\np7HVL/LXTnNjHY09+uhPg+Prli+jc9pa3gyOn3HzHKza8Oeckr8vP/afCGCdu7/u7u8DeBjA2X14\nPBHJo74k/xEANnb7vDk7JiL7gQG/4WdmM82s1sxq32wL/6giIvnXl+R/A0Blt88rsmMf4u5z3L3G\n3WsOLeG/h4tIfvUl+ZcDqDazkWY2DMD5AJ7sn9MSkYHW67v9AGBmZwC4FV2lvnnufmPS1x9QMMQL\n0iN6fby90bxxE40tW8bv2LZt4b+anDZ1SnB88pQJdE7Del7O62znpZylC5fSWO3S5TR2THW4lFZd\nwSsEixctorHZs/nd+ecWLqax+obw9926kV/fioTKQlnqQBob8uZrNPaZaeHnbN7c+XTOt+/k33Nm\nBz+PirJqGvvV00/Q2LrG8POZAj9WZ2cqOP7ZG2/Gq00bcrrb36c6v7s/BeCpvjyGiAwO/YWfSKSU\n/CKRUvKLRErJLxIpJb9IpPpU6ttbhR852CecNCkvx0qlwqWQnnR0dNDYWlK+amn7X3/b9D+P18nL\neSVFRTT2zGMLaCxdEF68AwA33RCutl5/3RV0TqqAF33a2/kqto0t/PtmhaTfLX6Rzkgqb/7Tl8+n\nsSXPPERjdcteIBG+sCcD/j3/8Ee/pLFrr7+Lxo6ZcCyNzfr+VcHxikL+R3GdW8Kv09P+4fOoa1gz\n4At7RGQ/puQXiZSSXyRSSn6RSCn5RSKV17v9oz4+2m/6yQ/ycqxUit/Brl3G2y01kDv6ALC6/o/B\n8Y0tf6Zzxk78OI1VVfKFLK3rN9BYe0JfhPvm3hMcv+a6a+ice+69l8Yybby1VjrN+xMeXnZYcLwo\nobJQWpCwaGYkb+O1bDm7ow+8ui7cPqsoYXl5UyNfFHb3vXwRVLqIP5+33nkrjZ173pnB8dKEqk7J\n8HCPxJ/cfyfeaB34Nl4ish9T8otESskvEiklv0iklPwikVLyi0Qqvwt7ig/2CafmZ2HP+BN5X72C\nTl5S2rKVL/gYU310ODCcP177zvdorLM9Q2Mp8Hkb14dLjgCwsi5c2rrjp/9B59x97wM0dswYXqoc\nO5YvVmF9EpvX8N1wzpzKdzAqrwyXtgBgwVO/orH6uvB5dCQs7GlLeA1UsdcAgPETxtJYRQUvA6aG\nhxehlZfxOceRa/8XE09GXe0KlfpEhFPyi0RKyS8SKSW/SKSU/CKRUvKLRKqv23U1AdgGYBeATnev\nSfr69EcK/YSTxvf6eHujpJiv2ioqKqGxZ57kq7a2ku21xiSUvNoyvB/cshd5P7vmRr66sCTN+xN+\n/aqLg+MLlvKtte6cez+NNa3npbn6+jU0dv+9DwfHv3pJuF8dAHz5sxfQWFJPxqQVnKxXXyph+69M\nB181ubWdr3JM2uptZMIKznQ6XMYsq+SrJgsKdgXHp076a6x8edXAb9eVdZq7t/XD44hIHunHfpFI\n9TX5HcCzZvaymc3sjxMSkfzo64/9J7v7G2Z2GIBFZrbW3Zd0/4Ls/xRmAsCwhN+zRCS/+vTO7+5v\nZP/dDOBxACcGvmaOu9e4e83QYUP7cjgR6Ue9Tn4zO8jMCj/4GMA0AKv768REZGD15cf+wwE8bmYf\nPM6D7v5M0oTSQw/DzC9e2odD5q668mM0VpXQDPLyr/By05f/PnzujSt5Wa64lJd4PnoQL0eO/SQv\nbW1qaqSxqZ+YEj7Wx3g58se3/ZTGRlfz67hoIS8fnjplcnD8weu/RudMmXwajXV2hEtbALCjcwiN\nVZLXQVGa/wpaUsqvfVPj6zTWtjWhRJjhKzjHjgk/N5kO3hi2bkV4taJZ7u/nvU5+d38dwAm9nS8i\ng0ulPpFIKflFIqXkF4mUkl8kUkp+kUjltYHnQYWFfty4/Kzqq1/O9+PrBF9p97slT9PYow/eHhy/\nbz5vIPmFf/gOjZUU8aaU447njTN/s5CvPHxiQXg13Tf//TY6p72jg8ZW14cbgnbhJbaLLv9KcPyY\nMaP4eSSUypYl7K84cWK4vAkAk08KxyoSmmO2tfK9+lIJew2m03xvvaaEvReLismqvjJeCm4nqwtf\neakW7e+8owaeIsIp+UUipeQXiZSSXyRSSn6RSOX1bn95ZYVfeOXleTlWuvhIGsts4X3Ypnyc32Et\nzIQrAQ0NvJfdiMO/RGMAX0AC0tcNAO57ZAGNXXZVuEfeN2+6hc4ZN4bf+Z4xlW9BVTKS90JsLwif\nf0HCNlmdbfwue2nC3fmScr5Qq6QyHFu9hm95tmUdf33MOJ1vKVZdzSsZbHstACgYGq4gLH5+KZ2z\nrjF8/mdOPQmr6l7W3X4R4ZT8IpFS8otESskvEiklv0iklPwikeqPHXtytnlzG36U0C+uP7XzihLW\nNfAyz8y/5n3kLjsrXG6qKjuaznl+FV+Q0rGTl/p2gPeYW/richorKglvvfX9WdfSOc31y2hsbDEN\n4bjKw2js6d+HFwRlwjueAQBeWBLuSwcAx0zgpbKHHuMLq34zNLwl2t/9LX/OxlVPpLH2rb+ksUUL\n+aK1ipFH0dhasu3ZWWfz107TxnAvwffff4/O2ZPe+UUipeQXiZSSXyRSSn6RSCn5RSKl5BeJVI+l\nPjObB+BMAJvd/bjs2CEAfgagCkATgPPcfWtPj1VRUYFrbvpBX843d2leRlv8PK83ff1bvEZYSMor\nD80Pl9cA4MePv0Bj370x3BMQAJbW8y3A2hPqZd+4NlyOLEjx6/Hdf+X9/Tat4T0NvzfrGzR28ZXh\n3oWjx42jcyZN5WXWdQml27b1b9BYVTlZDZjhfQszGb6qj/Xb68l98/hrZNHCcE4k9Zpc2/DR4Phb\nb/E+iHvK5Z3/HgDT9xi7GsBz7l4N4Lns5yKyH+kx+d19CYC39hg+G8D87MfzAZzTz+clIgOst7/z\nH+7uLdmPW9G1Y6+I7Ef6fMPPu1oB0XZAZjbTzGrNrLb9nXf6ejgR6Se9Tf5NZlYOANl/N7MvdPc5\n7l7j7jXpgw/u5eFEpL/1NvmfBHBB9uMLADzRP6cjIvmSS6nvIQCnAigxs2YA3wZwM4BHzOxCABsA\nnJfLwTo6OrC2kZew+lMZadwIAPX1S2hsxsTjaWxoZ3ilXWo4by552ZWX0NgOXm3CS7V8pV3dqzz2\nxDNfCI5fdO11dM62wk/Q2PZdfDXay2v4fd4f3/54cPz4o6fROYcfzI91w6xZNLb1G/waV48Nb3tW\nkNAgNV3IXzupFF9tmSFbaHXhtcqVdeHVez9//AE6p6oq/Jpb8tSvE87hw3pMfnf/HAl9KuejiMg+\nR3/hJxIpJb9IpJT8IpFS8otESskvEqm8NvD03Y7OjoT6Vj+6+/Z7aKylha8Ca2/hDRA/c9aZwfEX\nt/OVVI/eOpfGnl64mMbGTOV7wv3ns3yvvtOnzQiON9T/jM4pTIf39wOAOxbzJpIjr+f7Gi789c+D\n4xUJjTgXPM9LmEVzw48HAEcdPZXGXnnlteD4jL85hc6Ze8+tNFaRsGdgcyvfa7BhDS9x//LxcAPS\nhjXhVaQAcM654dfim39qpHP2pHd+kUgp+UUipeQXiZSSXyRSSn6RSCn5RSKV11JfOj0CkyZPyMux\njhvLS0rpdJrGOjv56quC4eEVXVPO2rPF4f+46PKLaez7t/AGnnfczhs+XnUZX+E267rZwfFjj+Sr\nFRc8xsuvjQ030tin/yZcbgKAL33h3OD4dxKahT47jZc36xL2V/zuLfwx2Qv8mDHP0Dmz/vVKGrsu\noWnpgQlNUn+7eCmNbWsPN+r8x0u+TOdUVoZLjpkDhtA5e9I7v0iklPwikVLyi0RKyS8SKSW/SKTy\nerd/xIgRGD9hfF6OVVLCF520teW+pVF3KXI3d9lL/M7xmDHhHnIAUFHBe8VN/fWnaWzXe+E7+gBw\n+Vf/OTieySQ81QkVjnTCHeyS0hIam/WtcJVgdeOf6JxU6nIa+84N36OxTBvfvqyoqCg43ryRL+46\nbRpfRJT63o9orPwOvhVd1cgj+WOSa5wuTHjtkLv9m15fTefsSe/8IpFS8otESskvEiklv0iklPwi\nkVLyi0Qql+265gE4E8Bmdz8uO3Y9gIsAbMl+2TXu/lRPj7V71250bOM98vpTR5ofh5VWgOSFPc2k\n99/SJS/SOadPO6lXx2q6i5fEhhrvB1dcHC5xbtuWsF3U8joa+8xZfAutti1jaWzBk+Hy5+SpvLy5\nfDkvbWUyfF5ZGb/GK+vC33dSv72W1sk0Nuv6m2jsqf/kJcKyspNprIP0tSwcexCdw0rZBQW5V+9z\neee/B0Bo2doP3X1c9r8eE19E9i09Jr+7LwHwVh7ORUTyqC+/819mZqvMbJ6ZFffbGYlIXvQ2+e8A\nMArAOAAtAOjfNZrZTDOrNbParVv1A4TIvqJXye/um9x9l7vvBnAngBMTvnaOu9e4e01x8SG9PU8R\n6We9Sn4zK+/26bkAcl9NICL7hFxKfQ8BOBVAiZk1A/g2gFPNbBwAB9AEgDeq62bYsANRUclXN/Wn\nklK+qq+0jJeNdnSE+6kBQFPjhuB4Q8Of6ZxUwRoaKz/slzT2+GN8K6/xx/Oee5m2cKyoJLy6DQAm\nJmyhddk/f43GtnTw1YydqXCZqp5cQwCYPO0rNPaX007jx+rk/Q4Xbwmv4CwspFPw0APh7bMAoLiQ\nl9+mz+A9CI8Yw3soHkNWflZkeDkSYKVbT5jzYT0mv7t/LjDMN6ATkf2C/sJPJFJKfpFIKflFIqXk\nF4mUkl8kUuaee2mgr4alRnhpVXVejlVWzst5SSufioo/QmMdO8JltOZm3hC0LKGsuG49X03Xtok3\npVzxX5tpbPHzq4Ljra28xPZPl3yRxsZN4NurPbuQlyNZg8mJE3kD1y2kLAcAzc284WbSKk0m6XlZ\nWRe+hgCwfj1fbVldPYrGPvvFv6Sx2beGVwqWlPPXYs3EcHn23ZY3seu9nUYndqN3fpFIKflFIqXk\nF4mUkl8kUkp+kUgp+UUilde9+g4cnsLohL3r+lNRYZrG9qbJYXcjq8LNLCeeyFcQtmzhK7NGj+Er\n7S744gU0dtGlfF3VJV8JrzrLZHg5rIg0/QSAkVV8FWZSuSyVOjA43tDAm4+uTYg1b+TPWVIj1JqJ\n4VLlaVN5CbOsnD9nBQW8nJe0kjTp+77o4vBzndnKS5+r614Pjp85lTcK3ZPe+UUipeQXiZSSXyRS\nSn6RSCn5RSKV37v9Bw7D6Gq+/VN/qiwPLywB+PZIALBxI7/Tu4PM29bO56xr/CM/j07e++/z59MQ\n6lbU09gistimveNtOmfGGbz33MZWvpBl6VK+TdmVV301OF5UxBerLF7M+yc2rHmNxjIZ/r29sDR8\nB37SZNpwGq1b+eO1t/NzbE3YAuzW2ctp7FPTx5Fj8S3nHnrw6eD4W2/yc9+T3vlFIqXkF4mUkl8k\nUkp+kUgp+UUipeQXiVQu23VVArgXwOHo2gtojrvfZmaHAPgZgCp0bdl1nrtvTXqsIQcYCoeHF3z0\nt1R6CI2l07zcVDWSlwhT6fC5NzXyclhbG+8917SRL0gZP34ijc276z9o7N23yYKmTl4CevBnfEux\nlS/yct6jj/DvezmZl05YRFRXz8uibRm+yCWpdNu0Ivy97Vz2ezpnbCFffLaxPqmXIH8+7573MI1t\nfqs5OD558mQ6p3NneHxvWnLm8s7fCeAqdz8WwCQAl5rZsQCuBvCcu1cDeC77uYjsJ3pMfndvcfcV\n2Y+3AWgAcASAswHMz37ZfADnDNRJikj/26vf+c2sCsB4AC8BONzdW7KhVnT9WiAi+4mck9/M0gB+\nAeAKd3+ne8y7mv8Hf9sws5lmVmtmtTu2b+/TyYpI/8kp+c1sKLoS/wF3fyw7vMnMyrPxcgDBnSTc\nfY6717h7zfARI/rjnEWkH/SY/GZmAOYCaHD32d1CTwL4oP/QBQCe6P/TE5GBksuqvk8C+BKAejNb\nmR27BsDNAB4xswsBbABwXk8PtG1bOxYvWdrbc90rFSP5LYiiNO+dl1Q2YttCFReH++YBQFEJ751X\n0MIvfwH4vJV1D9DYjL/62+D47H+7is656Ts8dtmlSecxi8a+eVX4+v9mOd8Kq7mFl/OSVtMlSZPt\n1zLb+Iq5dY28397q+mU0VlTEy8QLnuSv+xNOaA0/Xgl/DY8+PtxLMDUi91J6j8nv7i8AYHt/fSrn\nI4nIPkV/4ScSKSW/SKSU/CKRUvKLRErJLxKpvDbw3O270dHRu5LN3mpLKBs1d/CVWUnNIDt2hMtD\nW9ra6JwpU3ijSNZsEwBO+eRnaWzYAXzevHnhMuD6Df9F58y/fxKN/eiuf6exuvoGGltbH15NN/2s\nT9M5d8//OY01b+TPWTrNt2ZjDUOb1vMVhGvreSPOcWP5Nl/jJ/BVeGVlH6OxHds3BMe/3hgeB4Dj\nisOPt3svlvXpnV8kUkp+kUgp+UUipeQXiZSSXyRSSn6RSOW11FdQMAQlJbx5Zn8qKe1dYyG2cg8A\n2tvbg+O/e5Gv9Eravy1pr7vt77C1VMC3bvwXGisoCJ//e7t5Geqd7cU09i+3zKOxO26/kMaObw43\n97zkCv68LOjFikoAKCnlTUFLysPHKyoKP5cAUFp8LI1VlPOS3aKF/PmceOJPaezUA8KNP2eUltA5\nZWTF39CC3FNa7/wikVLyi0RKyS8SKSW/SKSU/CKRyuvd/tGjq7HgyafzcqzUcL7YI5VK+rZ5bOTI\nI8MzyB12AFi/ni/OqKzgPd/eePM1Gnvw4dtpbHX968HxBeP5gqqvXXEDjY2pPprG7kzxa1y7NXzn\nfumTi+icto38WiU9L6mE619I7n7vGMrnJB1rymnn09gt/8Yf8ROf5FuAXXThxcHxsqr76ZxUYXi8\nsD3YRDtI7/wikVLyi0RKyS8SKSW/SKSU/CKRUvKLRMq8h55fZlYJ4F50bcHtAOa4+21mdj2AiwBs\nyX7pNe7+VNJjjR0/zp94fmGfTzoXHR3hxRJAcqmvo4Nv41RWdlhwPF3IS16rV/Gtn+rqVtBYUvmw\no4MvSkFn+HtLpxO2L0tYBFVKeuD1pHF9eGHP2gZewtxI5gDAM0/zEuG2bbyMWV4efs6qRh5F50yc\nzBdBnXzKKTQ2efJEGksN589nZcVHw3MKE7ZzI6HWNXV4791tfFVY98fI4Ws6AVzl7ivMrBDAy2b2\nwTPxQ3dPqG6KyL4ql736WgC0ZD/eZmYNAPhfp4jIfmGvfuc3syoA4wG8lB26zMxWmdk8M+OLwkVk\nn5Nz8ptZGsAvAFzh7u8AuAPAKADj0PWTwQ/IvJlmVmtmtW+18V76IpJfOSW/mQ1FV+I/4O6PAYC7\nb3L3Xe6+G8CdAIK7U7j7HHevcfeaQ0p4xxURya8ek9/MDMBcAA3uPrvbeHm3LzsXwOr+Pz0RGSi5\nlPpOBrAUQD2A3dnhawB8Dl0/8juAJgAXZ28OUv9vyAG+bPiBfTzl3KxeFd4uCgDKSF+33upI6D3X\n2sJ7+CX2nkv4KakpoSR23PHHB8dTKf546dRBNFaacI5JffWYgr3oMddd0nZdLQl9Evnx+Hm0bsnQ\nWElCX72k56wt4VfedDp8/ceMHUXnsG3vNqz8HTq2JTSA7CaXu/0vAAg9WGJNX0T2bfoLP5FIKflF\nIqXkF4mUkl8kUkp+kUj1WOrrTyMKD/bqcZPycqy2Lby0srX9bRorJmUXgK/MSiWUytrb+YqzoiI+\nL2nVWWvC98bKdm1t/HvOZHhp65gxvIFnWRkvmbLSVscOXhbt7OQrMTMZfv5J81g5srWljc7ZknCt\nKkfy7bq61sCFJZWDWTmyqIS/Pvqj1Kd3fpFIKflFIqXkF4mUkl8kUkp+kUgp+UUipeQXiZSSXyRS\nSn6RSCn5RSKl5BeJlJJfJFJKfpFIKflFIqXkF4mUkl8kUkp+kUgp+UUipeQXiVQue/WlzOz3ZvaK\nmb1qZjdkxw8xs0Vm1pj9V1t0i+xHcnnnfw/AVHc/AV178003s0kArgbwnLtXA3gu+7mI7Cd6TH7v\n0p79dGj2PwdwNoD52fH5AM4ZkDMUkQGR0+/8ZjbEzFYC2Axgkbu/BODwbrvytgLo361vRWRA5ZT8\n7r7L3ccBqABwopkdt0fc0fXTwP9iZjPNrNbMajt37uzzCYtI/9iru/3ungGwGMB0AJvMrBwAsv9u\nJnPmuHuNu9cUDB3a1/MVkX6Sy93+UjMryn48HMDpANYCeBLABdkvuwDAEwN1kiLS/8L7BH1YOYD5\nZjYEXf+zeMTdf2VmywA8YmYXAtgA4LwBPE8R6Wc9Jr+7rwIwPjD+JoBPDcRJicjA01/4iURKyS8S\nKSW/SKSU/CKRUvKLRMq6/jgvTwcz24KusiAAlABoy9vBOZ3Hh+k8Pmx/O4+j3L00lwfMa/J/6MBm\nte5eMygH13noPHQe+rFfJFZKfpFIDWbyzxnEY3en8/gwnceH/Z89j0H7nV9EBpd+7BeJ1KAkv5lN\nN7PXzGydmQ1a7z8zazKzejNbaWa1eTzuPDPbbGaru43lvSEqOY/rzeyN7DVZaWZn5OE8Ks1ssZmt\nyTaJvTw7ntdrknAeeb0meWua6+55/Q/AEAB/BDAKwDAArwA4Nt/nkT2XJgAlg3DcUwBMALC629j3\nAFyd/fhqALcM0nlcD+D/5/l6lAOYkP24EMAfAByb72uScB55vSYADEA6+/FQAC8BmNTf12Mw3vlP\nBLDO3V939/cBPIyuZqDRcPclAN7aYzjvDVHJeeSdu7e4+4rsx9sANAA4Anm+JgnnkVfeZcCb5g5G\n8h8BYGO3z5sxCBc4ywE8a2Yvm9nMQTqHD+xLDVEvM7NV2V8L8rofg5lVoat/xKA2id3jPIA8X5N8\nNM2N/Ybfyd7VmHQGgEvN7JTBPiEguSFqHtyBrl/JxgFoAfCDfB3YzNIAfgHgCnd/p3ssn9ckcB55\nvybeh6a5uRqM5H8DQGW3zyuyY3nn7m9k/90M4HF0/UoyWHJqiDrQ3H1T9oW3G8CdyNM1MbOh6Eq4\nB9z9sexw3q9J6DwG65pkj73XTXNzNRjJvxxAtZmNNLNhAM5HVzPQvDKzg8ys8IOPAUwDsDp51oDa\nJxqifvDiyjoXebgmZmYA5gJocPfZ3UJ5vSbsPPJ9TfLWNDdfdzD3uJt5BrrupP4RwLWDdA6j0FVp\neAXAq/k8DwAPoevHx53ouudxIYBD0bXtWSOAZwEcMkjncR+AegCrsi+28jycx8no+hF2FYCV2f/O\nyPc1STiPvF4TAMcDqMsebzWAWdnxfr0e+gs/kUjFfsNPJFpKfpFIKflFIqXkF4mUkl8kUkp+kUgp\n+UUipeQXidR/A3h/dmUoKR7nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f95d9a96d30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X_batch[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers.core import Dense, Dropout, Activation, Flatten\n",
    "\n",
    "from keras.layers.convolutional import Conv2D\n",
    "from keras.layers.pooling import MaxPooling2D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Create a ConvNet Model\n",
    "\n",
    "# number of convolutional filters to use\n",
    "nb_filters = 32\n",
    "# size of pooling area for max pooling\n",
    "nb_pool = 2\n",
    "# convolution kernel size\n",
    "nb_conv = 3\n",
    "\n",
    "shape_ord = X_train.shape[1:]\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Conv2D(nb_filters, (nb_conv, nb_conv), \n",
    "                 padding='valid',\n",
    "                 input_shape=shape_ord))\n",
    "model.add(Activation('relu'))\n",
    "model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))\n",
    "model.add(Dropout(0.25))\n",
    "\n",
    "model.add(Flatten())\n",
    "model.add(Dense(nb_classes))\n",
    "model.add(Activation('softmax'))\n",
    "\n",
    "model.compile(loss='categorical_crossentropy',\n",
    "          optimizer='sgd',\n",
    "          metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0\n",
      "50000/50000 [==============================] - 18s - train loss: 1.6592    \n",
      "Epoch 1\n",
      "50000/50000 [==============================] - 18s - train loss: 1.6445    \n"
     ]
    }
   ],
   "source": [
    "from keras.utils import generic_utils\n",
    "\n",
    "n_epochs = 2\n",
    "for e in range(n_epochs):\n",
    "    print('Epoch', e)\n",
    "    batches = 0\n",
    "    progbar = generic_utils.Progbar(X_train.shape[0])\n",
    "    for X_batch, Y_batch in generated_images.flow(X_train, Y_train, batch_size=500, shuffle=True):\n",
    "        loss = model.train_on_batch(X_batch, Y_batch)\n",
    "        progbar.add(X_batch.shape[0], values=[('train loss', loss[0])])\n",
    "        batches += 1\n",
    "        if batches >= len(X_train) / 500:\n",
    "            # we need to break the loop by hand because\n",
    "            # the generator loops indefinitely\n",
    "            break\n",
    "        "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
